{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# BIDMach: basic classification"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For this tutorial, we'll BIDMach's GLM (Generalized Linear Model) package. It includes linear regression, logistic regression, and support vector machines (SVMs). The imports below include both BIDMat's matrix classes, and BIDMach machine learning classes. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1 CUDA device found, CUDA version 7.0\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "(0.96885055,11703132160,12079398912)"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import BIDMat.{CMat,CSMat,DMat,Dict,IDict,FMat,FND,GDMat,GMat,GIMat,GSDMat,GSMat,HMat,Image,IMat,Mat,SMat,SBMat,SDMat}\n",
    "import BIDMat.MatFunctions._\n",
    "import BIDMat.SciFunctions._\n",
    "import BIDMat.Solvers._\n",
    "import BIDMat.JPlotting._\n",
    "import BIDMach.Learner\n",
    "import BIDMach.models.{FM,GLM,KMeans,KMeansw,ICA,LDA,LDAgibbs,NMF,RandomForest,SFA}\n",
    "import BIDMach.datasources.{MatSource,FileSource,SFileSource}\n",
    "import BIDMach.mixins.{CosineSim,Perplexity,Top,L1Regularizer,L2Regularizer}\n",
    "import BIDMach.updaters.{ADAGrad,Batch,BatchNorm,IncMult,IncNorm,Telescoping}\n",
    "import BIDMach.causal.{IPTW}\n",
    "\n",
    "Mat.checkMKL\n",
    "Mat.checkCUDA\n",
    "Mat.setInline\n",
    "if (Mat.hasCUDA > 0) GPUmem"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Dataset: Reuters RCV1 V2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The dataset is the widely used Reuters news article dataset RCV1 V2. This dataset and several others are loaded by running the script <code>getdata.sh</code> from the BIDMach/scripts directory. The data include both train and test subsets, and train and test labels (cats). "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": []
    },
    {
     "data": {
      "text/plain": [
       "2.314"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "var dir = \"../data/rcv1/\"  // Assumes bidmach is run from BIDMach/tutorials. Adjust to point to the BIDMach/data/rcv1 directory\n",
    "tic\n",
    "val train = loadSMat(dir+\"docs.smat.lz4\")\n",
    "val cats = loadFMat(dir+\"cats.fmat.lz4\")\n",
    "val test = loadSMat(dir+\"testdocs.smat.lz4\")\n",
    "val tcats = loadFMat(dir+\"testcats.fmat.lz4\")\n",
    "toc"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "BIDMach's basic classifiers can invoked like this on data that fits in memory:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": []
    },
    {
     "data": {
      "text/plain": [
       "BIDMach.models.GLM$LearnOptions@3ec51f5"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "val (mm, opts) = GLM.learner(train, cats, GLM.logistic)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The last option specifies the type of model, linear, logistic or SVM. The syntax is a little unusual. There are two values returned. The first <code>mm</code> is a \"learner\" which includes model, optimizer, and mixin classes. The second <code>opts</code> is an options object specialized to that combination of learner components. This design facilitates rapid iteration over model parameters from the command line or notebook. \n",
    "\n",
    "The parameters of the model can be viewed and modified by doing <code>opts.what</code>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Option Name       Type          Value\n",
      "===========       ====          =====\n",
      "addConstFeat      boolean       false\n",
      "aopts             Opts          null\n",
      "autoReset         boolean       true\n",
      "batchSize         int           10000\n",
      "checkPointFile    String        null\n",
      "checkPointInterval  float         0.0\n",
      "cumScore          int           0\n",
      "debug             int           0\n",
      "debugMem          boolean       false\n",
      "dim               int           256\n",
      "doubleScore       boolean       false\n",
      "epsilon           float         1.0E-5\n",
      "evalStep          int           11\n",
      "featThreshold     Mat           null\n",
      "featType          int           1\n",
      "hashBound1        int           1000000\n",
      "hashBound2        int           1000000\n",
      "hashFeatures      int           0\n",
      "initsumsq         float         1.0E-5\n",
      "iweight           FMat          null\n",
      "langevin          float         0.0\n",
      "lim               float         0.0\n",
      "links             IMat          1,1,1,1,1,1,1,1,1,1,...\n",
      "lrate             FMat          1\n",
      "mask              FMat          null\n",
      "momentum          FMat          null\n",
      "nesterov          FMat          null\n",
      "npasses           int           2\n",
      "nzPerColumn       int           0\n",
      "pexp              FMat          0.50000\n",
      "policies          Function2[]   null\n",
      "pstep             float         0.01\n",
      "putBack           int           -1\n",
      "r1nmats           int           1\n",
      "reg1weight        FMat          1.0000e-07\n",
      "resFile           String        null\n",
      "rmask             FMat          null\n",
      "sample            float         1.0\n",
      "sizeMargin        float         3.0\n",
      "startBlock        int           8000\n",
      "targets           FMat          null\n",
      "targmap           FMat          null\n",
      "texp              FMat          0.50000\n",
      "updateAll         boolean       false\n",
      "useCache          boolean       true\n",
      "useDouble         boolean       false\n",
      "useGPU            boolean       true\n",
      "vexp              FMat          0.50000\n",
      "waitsteps         int           3\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "0.30000"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "opts.what\n",
    "opts.lrate=0.3f"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Most of these will work well with their default values. On the other hand, a few have a strong effect on performance. Those include:\n",
    "<pre>\n",
    "lrate: the learning rate\n",
    "batchSize: the minibatch size\n",
    "npasses: the number of passes over the dataset\n",
    "</pre>\n",
    "We will talk about tuning those in a moment. For now lets train the model:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "corpus perplexity=5582.125391\n",
      "pass= 0\n",
      " 2.00%, ll=-0.69315, gf=1.439, secs=0.5, GB=0.02, MB/s=45.57, GPUmem=0.906281\n",
      "16.00%, ll=-0.06984, gf=6.724, secs=0.8, GB=0.13, MB/s=169.60, GPUmem=0.906281\n",
      "30.00%, ll=-0.05885, gf=8.968, secs=1.1, GB=0.25, MB/s=222.78, GPUmem=0.906281\n",
      "44.00%, ll=-0.05025, gf=10.087, secs=1.5, GB=0.36, MB/s=248.91, GPUmem=0.906281\n",
      "58.00%, ll=-0.04849, gf=10.703, secs=1.8, GB=0.48, MB/s=263.14, GPUmem=0.906281\n",
      "72.00%, ll=-0.03930, gf=11.182, secs=2.1, GB=0.59, MB/s=274.45, GPUmem=0.906281\n",
      "87.00%, ll=-0.04346, gf=11.537, secs=2.5, GB=0.70, MB/s=282.83, GPUmem=0.906281\n",
      "100.00%, ll=-0.03860, gf=11.681, secs=2.8, GB=0.81, MB/s=284.46, GPUmem=0.905934\n",
      "pass= 1\n",
      " 2.00%, ll=-0.04507, gf=11.722, secs=2.9, GB=0.83, MB/s=287.13, GPUmem=0.905934\n",
      "16.00%, ll=-0.03975, gf=11.876, secs=3.2, GB=0.94, MB/s=290.65, GPUmem=0.905934\n",
      "30.00%, ll=-0.04341, gf=11.886, secs=3.6, GB=1.05, MB/s=290.78, GPUmem=0.905934\n",
      "44.00%, ll=-0.03979, gf=11.392, secs=4.2, GB=1.17, MB/s=278.54, GPUmem=0.905934\n",
      "58.00%, ll=-0.04178, gf=11.367, secs=4.6, GB=1.28, MB/s=277.77, GPUmem=0.905934\n",
      "72.00%, ll=-0.03438, gf=11.455, secs=5.0, GB=1.39, MB/s=279.87, GPUmem=0.905934\n",
      "87.00%, ll=-0.04052, gf=11.499, secs=5.4, GB=1.51, MB/s=280.90, GPUmem=0.905934\n",
      "100.00%, ll=-0.03577, gf=11.575, secs=5.7, GB=1.61, MB/s=281.87, GPUmem=0.905934\n",
      "Time=5.7200 secs, gflops=11.58\n"
     ]
    }
   ],
   "source": [
    "opts.npasses=2\n",
    "mm.train"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The output includes important information about the training cycle:\n",
    "* Percentage of dataset processed\n",
    "* Cross-validated log likelihood (or negative loss)\n",
    "* Overall throughput in gigaflops\n",
    "* Elapsed time in seconds\n",
    "* Total Gigabytes processed\n",
    "* I/O throughput in MB/s\n",
    "* GPU memory remaining (if using a GPU)\n",
    "\n",
    "The likelihood is calculated on a set of minibatches that are held out from training on every cycle. So this is a cross-validated likelihood estimate. Cross-validated likelihood will increase initially, but will then flatten and may decrease. There is random variation in the likelihood estimates because we are using SGD. Determining the best point to stop is tricky to do automatically, and is instead left to the analyst. \n",
    "\n",
    "To evaluate the model, we build a classifier from it:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": []
    },
    {
     "data": {
      "text/plain": [
       "BIDMach.models.GLM$PredOptions@3aa5218c"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "val (pp, popts) = GLM.predictor(mm.model, test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And invoke the predict method on the predictor:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "corpus perplexity=65579.335560\n",
      "Predicting\n",
      " 3.00%, ll=0.00000, gf=0.053, secs=0.3, GB=0.00, MB/s= 1.82, GPUmem=0.95\n",
      " 6.00%, ll=0.00000, gf=0.103, secs=0.3, GB=0.00, MB/s= 3.56, GPUmem=0.95\n",
      "10.00%, ll=0.00000, gf=0.147, secs=0.3, GB=0.00, MB/s= 5.02, GPUmem=0.95\n",
      "13.00%, ll=0.00000, gf=0.198, secs=0.3, GB=0.00, MB/s= 6.80, GPUmem=0.95\n",
      "16.00%, ll=0.00000, gf=0.245, secs=0.3, GB=0.00, MB/s= 8.41, GPUmem=0.95\n",
      "20.00%, ll=0.00000, gf=0.290, secs=0.3, GB=0.00, MB/s= 9.91, GPUmem=0.95\n",
      "23.00%, ll=0.00000, gf=0.330, secs=0.3, GB=0.00, MB/s=11.26, GPUmem=0.95\n",
      "26.00%, ll=0.00000, gf=0.372, secs=0.3, GB=0.00, MB/s=12.71, GPUmem=0.95\n",
      "30.00%, ll=0.00000, gf=0.418, secs=0.3, GB=0.00, MB/s=14.29, GPUmem=0.95\n",
      "33.00%, ll=0.00000, gf=0.464, secs=0.3, GB=0.00, MB/s=15.87, GPUmem=0.95\n",
      "36.00%, ll=0.00000, gf=0.511, secs=0.3, GB=0.01, MB/s=17.49, GPUmem=0.95\n",
      "40.00%, ll=0.00000, gf=0.554, secs=0.3, GB=0.01, MB/s=18.93, GPUmem=0.95\n",
      "43.00%, ll=0.00000, gf=0.595, secs=0.3, GB=0.01, MB/s=20.35, GPUmem=0.95\n",
      "46.00%, ll=0.00000, gf=0.656, secs=0.3, GB=0.01, MB/s=22.49, GPUmem=0.95\n",
      "50.00%, ll=0.00000, gf=0.696, secs=0.3, GB=0.01, MB/s=23.89, GPUmem=0.95\n",
      "53.00%, ll=0.00000, gf=0.738, secs=0.3, GB=0.01, MB/s=25.29, GPUmem=0.95\n",
      "56.00%, ll=0.00000, gf=0.782, secs=0.3, GB=0.01, MB/s=26.81, GPUmem=0.95\n",
      "60.00%, ll=0.00000, gf=0.824, secs=0.3, GB=0.01, MB/s=28.27, GPUmem=0.95\n",
      "63.00%, ll=0.00000, gf=0.864, secs=0.3, GB=0.01, MB/s=29.61, GPUmem=0.95\n",
      "66.00%, ll=0.00000, gf=0.905, secs=0.3, GB=0.01, MB/s=31.03, GPUmem=0.95\n",
      "70.00%, ll=0.00000, gf=0.946, secs=0.3, GB=0.01, MB/s=32.43, GPUmem=0.95\n",
      "73.00%, ll=0.00000, gf=0.986, secs=0.3, GB=0.01, MB/s=33.81, GPUmem=0.95\n",
      "76.00%, ll=0.00000, gf=1.021, secs=0.3, GB=0.01, MB/s=34.98, GPUmem=0.95\n",
      "80.00%, ll=0.00000, gf=1.066, secs=0.3, GB=0.01, MB/s=36.56, GPUmem=0.95\n",
      "83.00%, ll=0.00000, gf=1.106, secs=0.3, GB=0.01, MB/s=37.92, GPUmem=0.95\n",
      "86.00%, ll=0.00000, gf=1.146, secs=0.3, GB=0.01, MB/s=39.30, GPUmem=0.95\n",
      "90.00%, ll=0.00000, gf=1.180, secs=0.3, GB=0.01, MB/s=40.43, GPUmem=0.95\n",
      "93.00%, ll=0.00000, gf=1.224, secs=0.3, GB=0.01, MB/s=41.99, GPUmem=0.95\n",
      "96.00%, ll=0.00000, gf=1.262, secs=0.3, GB=0.01, MB/s=43.28, GPUmem=0.95\n",
      "100.00%, ll=0.00000, gf=1.300, secs=0.3, GB=0.01, MB/s=44.61, GPUmem=0.95\n",
      "Time=0.3150 secs, gflops=1.30\n"
     ]
    }
   ],
   "source": [
    "pp.predict"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Although ll values are printed above, they are not meaningful (there is no target to compare the prediction with). \n",
    "\n",
    "We can now compare the accuracy of predictions (preds matrix) with ground truth (the tcats matrix). "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": []
    },
    {
     "data": {
      "text/plain": [
       "    0.051716  2.8286e-15  9.3898e-10  9.3898e-10  0.00024929  8.0685e-12...\n",
       "    0.011135  3.4162e-05  4.7918e-06  4.7918e-06   0.0059853  9.2513e-08...\n",
       "     0.78789  2.8876e-12  1.7810e-09  1.7810e-09  1.3463e-05  5.0863e-09...\n",
       "    0.011328  2.3973e-15  1.1961e-11  1.1961e-11  3.4776e-05  8.7643e-06...\n",
       "     0.99887  1.0619e-06  2.1314e-09  2.1314e-09  2.3170e-05           1...\n",
       "  2.0127e-07     0.99854  7.5009e-07  7.5009e-07     0.14460  3.9604e-07...\n",
       "  1.1595e-05           1      1.0000      1.0000     0.90988  1.4346e-05...\n",
       "  3.8421e-07  7.1216e-06     0.73840     0.73840    0.062895  4.6554e-11...\n",
       "          ..          ..          ..          ..          ..          ..\n"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "val preds = FMat(pp.preds(0))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": []
    },
    {
     "data": {
      "text/plain": [
       "-0.039590"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "val lls = mean(ln(1e-7f + tcats ∘ preds + (1-tcats) ∘ (1-preds)),2)  // actual logistic likelihood\n",
    "mean(lls)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A more thorough measure is ROC area:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": []
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfQAAAGQCAYAAABYs5LGAAAyZ0lEQVR42u2da3BV5bnHnel0Op1OP/RLP3Q6nU6/dXp65pxpj6dzestQMuFAObZwCERBMdFIo9AEjVzCJYdYi5eamAgFsVw0TdMgAUVRBLEVFISUgCIWFRoFEUhMBISCxDxn/1fc2703O8m+hbwr6/ebWSbvziL5+7x7r//zvM+6XGUAAADge64iBAAAABg6AAAAYOgAAACAoQMAAACGDgAAgKEDQO+H6Kqrkt4S7R8kfvnLXwb2/x0AQwfA0H3PiRMnrLq6OtDJDACGDjDMDJ34YOgAGDqADwwLMHQADB0gAIbe1/6XLl2yyspK+/rXv26f//zn7Ytf/KLl5OR43/f1+/v6Xf1piv/ZJ598Ynfffbd985vftM997nOR/V577TW76aab7Bvf+EZEz3e+8x2bN2+enT17FkMHwNABMPRE++fl5aW8bJ8NQ//Rj3502b6rV6/2zL0vDTL2zs5OVjMAMHSA4Wvo6ZwUV19ff5nJdnR02KlTpwbd0OO31tbWmPGsWbO8Kr6uru6y1zF0AAwdAEOPei2+Ot+yZUta5pzOv2loaIhZQp88eXLMzy9evOi9LlOPfv1b3/oWhg6AoQNg6NGvqT8d39e+UoYez1e+8pWk/l/UV8fQATB0gGFr6Onsn64BD4ah99c7z9SQMXQADB1gWBt6fIU+lIb+5S9/OebnOvt+qGIFABg6gK8M/eqrr4557dChQ0n9/vhL2nbs2GEtLS0ZGfr48eNjfv7oo49ets+5c+e8S+owdAAMHQBDj9pf159Hv6brv9VHf+aZZ/r9/d/+9rezdqlbmN27d8f8/Ktf/ao1NjZ6lbo0bdu2zbtsLVlD5i56ABg6QGAMXdd06wYuA5le9I1fxKJFiy7bR2efZ2LoQpfRfeELX8iKCWPoABg6QGAMXRw7dsyuv/56+9KXvuQtpf/kJz+xF198MWZf9drjKSsr816XARcWFnrJQaaGLtra2qy8vNxrB+j3K5nQ3/jud79rxcXFXiWPoQNg6ACQBK+//nqM6aXTtwYADB0AriC6h/uSJUu8u8OJN954w6vSow1dS+EAABg6gMsfxgGWpKdOnUqQAABDB3Cda6+91r73ve9514GrV63ta1/7mk2aNMk2btxIgAAAQwcAAMDQAQAAAEMHAAAADB0AAAAwdAAAAAwdAAAAMHQAAADA0AEAAABDBwAAwNABAAAAQ3eN1tbWmPHNN99sCxcudHb79a9/7bQ+NBJDNBJDNA6+vtLSUgw9nr/85S8x4xtuuMFpvS+88ILzMUUjMUQjMUTj4OqTqWPoAxi6MimX+cc//uF8TNFIDNFIDNE4uPow9CQMPdUgAQAAXGkwdCp0NFK5oRF9aKRCD4ah00NHI/rQiD40uq4PQ6dCRyOVGxrRh0Yq9GAYOj10AABwHQydCh2NVG5oRB8aqdCDYej00NGIPjSiD42u6/O1obe1tdno0aP73aenp8dqa2utqqrKKisrva2rq4sKHY1k9GhEHxqp0F0gJycnsvVHfX29NTU1RcaNjY1WU1OTkqHTQwcAANfx/ZL7QIZeXFxsHR0dkXF7e7sVFBRQoaORjB6N6EMjFbqfDD0vLy9mrCX4+NfC6KEsMnNV8dFMnDjR63eEJ0BfXRqvX7/eaX36umrVKqf1aaw4oi+zcfxXPi8Z6gsdr9556y3765Yt1vbmm2YXLti7Bw7Y9meesXf+/nez8+e98Y5Nm7yv1tlp7738su1Zt86OtrSYHT9ux1980fb+6U/e63b4sJ0I7XtgxQp7//nnzV591U6F/t6hujo7EfqdFvo37c3N9vYDD9jJp54y273b2kO/6/Dvfmcnn3zSLPQ72pua7MCsWd5+FvrdHWvX2rGqKm+/0AHU2v/8ZzseMhbtZ9u2eV/fD407Qhps61bv6/vz5tkHjz5qFvqbH9TX24mKCuv8wx/MNm60zjVr7OTcudb1+9+bbdhgnaFjh37+4ZIlkfH78+fbh7W1kbH+3mmtvIY06fccX7DA2mbPNgv97a7ly739P7rrLi3Z2oeh/9eTod93PrRZaN8z1dV2as4c++edd5qF9j1z3312KvRvL+hBJ0uX2tl777WToZ9fnDEjMn4/9G8/nj7dLKTp7OLFnr5L06aZhTR99NvffjZ+8MHI+OOSErNQXMPjd8vLzT74IKvvn2Fv6Il+PtC/oUJHI/oc0hgyNevuNvv4Y7PTp83OnDHTeTDt7WZvvGF25IjZu+/qpBrPkGz/frNDh3p/tn272Y4dytZ7f/b00/ZByDBkRDIbC5mPaRwyElPC9PDDZvffb/bHP5o99phZ6OBv//d/ZiEDtGXLzEIHe7vjDrOQCVjI5Ow3vzHTgfruu80WLTILGZXdeKNZyJBMhqKD9nXXmd1++2ebxmVlvZtMQ2Md/G+6SWfgmuXnq4owmzCh93ttWlXUfpMn9/5MX6+/vnebNKl3XFSkJcne36Hvb7vNTCak7/W7pTtkxCYj0s908JdmvSYd+n8ImZOFzNDTFjIub9M++j2KS8iQ9NrH+vcyVG16TcdIxeqhhzyT88Yh84tsGsugFUPFWOap72XiK1eahQxXZurNhYw+lCB4+4UM2NukTfuHx/fc0/t3wmNpk5aGht4tpOVsyDhl6BZKOLzf9WlCYEpMNL+aUyUxzz5rpmREWkKJk5IOUzITShS894g27aefh8fPPderVye2yS/++lezRx7pfV+FN431engc/n2fjk/pb547R4U+WBV6X4ZODx18ySefmH30kVe12Xvvme3Z07vJ6A4e7DU6barstO3c2XsA2rev1/x0cNNBRwchHfRkcDIyHSS1iqUDlqoeHYR1sNKBO1TJeAdPHex0gJ8587ODvsxCxqKDtUxSxiHzkaHIbGQq116r5xWbTZ3aa2BhQ5Np6WcyM5mY9vnVr3r/vb7q7+h3yNBkHqq+ZBIyL5mojFd/X/pkutIkvXot/P8QPsjrZzL3J57o/f+UkWze3Hug14FdBvLSS73xUvwUo717vWrXXnvNTJWvqum33/YqYnvlFbN33vlsC4+PHjU7dqz335040Zuk6AD/z3/2zh0Ahh5LSSh77tQB7VPUQ8/XAYIKHY3paNSB9sKFXpPUAfn48d7KUAdlGaAO6PpehqgsXwaqA78O8jI6ZfabNvUahoxE5qiKQmYpY5NJaj+ZjQxOVaDMMFSBXLrlFrPKyt6KSWYoc9Q+MjN9ldEVFvaaocxPFZ5MMWxyqgL1VUanf69/o02/U5t+jyox6QpXatIkLdKlCksVmzQ//nivAcrgVd2GKl8ZoJYjvf9HxUJJgP5fVJFouVdGJvNTBa04KanQfm+91RtDmZsqa53zokTk4sXe6pyVGDQyz8E19OjXGkIHy+ieuL6v1kEoBUPnOnQHNV665PUP7cMPzU6d6jWJ11/vNdPdu3vNRstrMk991ZyHl+xWr+6t2MLLdg8+6FVuZ1XpyexU6WmZcsqUXoPU/Msgw0uh4eVOfZUBan9tMlxVnmHD1Vh/I2zKMkwZnJYYtayofWSO6kWqGtSyoYxRhqiqWPspbjLDXbvsiJZ7Q1+9/0eZoapE9VFV7ckMVRWq0pMZanma96IvNRJD5jlwhh592Vr85WvR33Md+hCjCuv0aTuupVz1OrW0q2VKLW2qglM/S31MLZHqNZmrqkNVmeEeZHgpVhWpXlc/UdWnDFWvqYcp01XfT4aqKlGVpYxUpigTl6GrByZzV+UsU1eipw+V+qwh4+/QWEapeMokVX2rXyuDVPKgipF5RiMxRCMVun+ghx6HTkRSBagTj1QRKj6qKrVcqspSVaeMVGeZqj+qBEhmK5MN9zxlyDJnLe3KsGXSWsZVpSpD10kv6lNqeVbL1TJ7Lb/qTa+lbRlteClW1TkAAGDoqRr6sKnQdbLNyZOfncyjTSYaXnYOnySkM1hV+WoLn4wkU9aSsU4o0olFn17i4RmyTijScrZ+n06o0tKv+qI6wefTHigZPVURGokhGqnQh9zQfdND15Kw+sjq28qg9VXmq16wlqlVNWt5WidHaTlbFbWqa/V3w5dkRJ8Jrf60LhfKpkY/xBF9aEQfGn2oD0P3c4Uuw33lFbugpWz1jGXYuv5V11/qLGqdHKZqWWcQqwdMtkxGj0bmGY1U6EE2dGd66DJw9Zh1ZvWtt/bewELmrRO+1Hce4pO2AABg6MDQXa/QdWa4Lm9Sz1pneOvSJi2P63X602T0aGSe0cg8Y+jJG/oV76FronUWuU5CU89b1zXrRhx99LPpZwVDIzFkntHIPGPofqnQdVmYrp3WmeU6WU2XiCVxpyyyZTJ6NDLPaGSeMfQkDH3Qe+g6M11L6bo/tW60osvLAAAAMHSfVOiqwHXfbFXkuh92mie1kS2T0aOReUYj84yhJ2HoWe+h6zaoCrxu4KL7dmd4djr9rGBoJIbMMxqZZwzdlQpdtyzVJWa6Xlz3D8/SIxLJlsno0cg8o5F5xtCTMPSs9NB1Dbme5KVeuR6/CQAAgKH7rEJXJqbLz/QQkgA+3xmNxBCNxBCNVOhOGHpGPXT1y/VwEz0QZZCgnxUMjcSQeUYj84yhD1WFrvun6xateswo2TIaiSEaiSEaqdCH1tDT7qHraWZ65jcAAMAgg6EPVoWu36Fbt378MdkyGokhGokhGqnQE9HT02O1tbVWVVVllZWV3tbV1ZVw3+7ubqurq7NRo0bZ6NGjbenSpSkbeso99BMnzIqKzN5554rEg35WMDQSQ+YZjczzsDP0+vp6a2pqiowbGxutRsvbCVi2bJnt3bvX+/706dO2ZMkSW7NmzeBV6Oqb68lozzxDJopGYohGYohGKvT+KC4uto6Ojsi4vb3dCgoKEu6rqjwaVfJTpkxJydBTCtLq1WbTphkAAMCVxJeGnpeXFzPWEnz8a2FGjBjh/Tya3NzcwanQ9XdmzDB74w0yUTQSQzQSQzRSoQ9ETk5OUq+J2bNne8vz6qXL2Pfs2dOnobe2tnpmriX8aCZOnOj1O8IToK+Jxic2bdIf7PPngzVev379Ff176YxXrVrltD6NFUf0ZTaO/+qiXtc/L3ye+TynOx72FbqW2OfPn++ZuE6Mq66utsm6l/pgVOiLFpm9+CKZKBqJIRqJIRqp0JOhpKTEOjs7I2P10PPz85P6t7t27fLOkE/F0JMKUlub2S236LR6AwAAuNL40tAbGhpilsX1vSrvMH0tvx84cMAKCwu9BCDrFbouh2tuJhNFIzFEIzFEIxV6sgx0HXq8oWu5XVtZWZkdOXJkwN+f8nXoepLa1KlmZ88OSTy4JjQYGokh84xG5nnYGfpgk3KFrmviH36YTBSNxBCNxBCNVOguG3q/QdKtXW++2ezYMQMAABgqMPRMK/Q//KH3WedkomgkhmgkhmikQnfb0Pvtoa9cafb440Oql35WMDQSQ+YZjcwzhj6YFfq8eWavv04mikZiiEZiiEYqdNcNvc8g6Zpz3aRGD2QBAAAYQjD0TCp03UymtJRMFI3oQyMxRCMVuh8Mvc8e+vPPmz300JDrpZ8VDI3EkHlGI/OMoQ9Wha5rz/VAFjJRNKIPjcQQjVTo7ht6n0GaNcvszTcNAABgqMHQ063QdUMZnRCnr2SiaEQfGokhGqnQ3Tf0hD30t982Ky93Qi/9rGBoJIbMMxqZZwx9MCr0Z581W7aMTBSN6EMjMUQjFbpfDD1hkPS41C1bDAAAwAUw9HQr9JkzzZJ4FCuZKBqJIRqJIRqp0B0x9Mt66Bcu9J4QpzvFOQD9rGBoJIbMMxqZZww92xX6G2+YzZ1LJopG9KGRGKKRCt1Phn5ZkJ56yuyRRwwAAMAVMPR0KvQHH9ROZKJoRB8aiSEaqdAzoaenx2pra62qqsoqKyu9raurK+G+7e3tNn/+fMvLy/O28vJyO378eEqGflkPfcYMs3ffdeaNQT8rGBqJIfOMRuZ52Bl6fX29NTU1RcaNjY1WU1OTcN/CwkJ74oknvCRA27p16/p+2EoyFfpHH5ldf72yCjJRNKIPjcQQjVTomVBcXGwdHR0xVXhBQUHCfXNzcz0jj2bkyJEpGXpMkPbsMauoMAAAAJfwpaFr6TwaGXb8a2Gam5ttzZo1tmnTJjt58qRt27bN7rnnnvQrdPXP58whE0Uj+tBIDNFIhZ4pOTk5Sb0mZOLTp0/3THzChAleP/3MmTMJ921tbfXMXEv40UycONHrd3gT8OST9mFd3WfjTydmKMfr1693Sk+i8apVq5zWp7HiiL7MxvFfXdTr+ueFzzOf53THw75CLykpscOHD0fG6r2XlpamX6FrEv74RzJRNKIPjcQQjVTomSKT7uzsjIzVQ8/Pz0+4b6J+eUY99Mcf11l4BgAA4BK+NPSGhoaYZXF9X11dHRlHL7/rMjUtu4cr+ZUrV1rFACe19Vuh6+/K1MlE0Yg+NBJDNFKhZ8ZA16FHG7pe1891trs29dL76qH3Zegxl7mFkglrbnbqjcE1ocHQSAyZZzQyz8PO0Aebfiv0Rx/1TowjE0Uj+tBIDNFIhe4zQ48J0qpVZk8/bQAAAC6BoadaoeuhLM8+SyaKRvShkRiikQrdb4Ye00Nfvtxsyxan9NLPCoZGYsg8o5F5xtCzWaEvXWq2bRuZKBrRh0ZiiEYqdL8ZekyQ6urM/vpXAwAAcAkMPdUKXU9127GDTBSN6EMjMUQjFbrfDD2mh37//Wa7djmll35WMDQSQ+YZjcwzhp7NCv3ee8127yYTRSP60EgM0UiF7jdDjwnS3Xeb7d1rAAAALoGhp1qhV1WZ7d9PJopG9KGRGKKRCt1vhh7TQ6+sNDtwwCm99LOCoZEYMs9oZJ4x9GxW6PPnm73xBpkoGtGHRmKIRip0vxl6TJDmzjV7800DAABwCQw91Qr9zjvNDh8mE0Uj+tBIDNFIhe43Q4/pod9xh1lbm1N66WcFQyMxZJ7RyDxj6Nms0EtLzY4dIxNFI/rQSAzRSIXuN0OPCdL06Wbvv28AAAAugaGnWqGXlJidOkUmikb0oZEYopEKPVN6enqstrbWqqqqrLKy0tu6uroS7puTk5NwS8XQY3rot9xi9sEHTr0x6GcFQyMxZJ7RyDwPO0Ovr6+3pqamyLixsdFq9BS0JOjo6LCpU6emX6HfdJPZhx+SiaIRfWgkhmikQs+U4uJiz5jDtLe3W0FBQVL/trq62rZs2ZKSoccE6cYbzc6eNQAAAJfwpaHn5eXFjLUEH/9aIo4fP25FRUUD7tdvhX799Wbnz5OJohF9aCSGaKRCz5REPfCB+uJi8eLFtn379j5/3tra6pm5lvCjmThxotfv8Cbg2mut7a23Pht/OjFDOV6/fr1TehKNV61a5bQ+jRVH9GU2jv/qol7XPy98nvk8pzsOTIXe1tZm06ZNS+r391uhT5pk1t1NJopG9KGRGKKRCj1TSkpKrLOzMzJWDz0/P7/ffzN//nzbm+RzzPvtoQ/wdwAAAIYCXxp6Q0NDzLK4vtfJbmHil98PHz5sM2bMSPr391mhqzJXhU4mikb0oZEYopEKPXMGug493tDLysrs4MGDaRt65Dr0ixfNrrvOuTcG14QGQyMxZJ7RyDwPO0MfbPqs0HV2u85yJxNFI/rQSAzRSIXuP0OPBOnMGbPCQgMAAHANDD2VCl13iNOd4shE0Yg+NBJDNFKh+8/QIz103Z0uyUvfriT0s4KhkRgyz2hknjH0bFXoJ0+a3XormSga0YdGYohGKnQ/GnokSMePm6Vw+RsAAMCVAkNPpUI/elTXwJGJohF9aCSGaKRC96OhR3robW1md9zhnF76WcHQSAyZZzQyzxh6tir0w4fNZs0iE0Uj+tBIDNFIhe5HQ48E6c03zebONQAAANfA0FOp0HX72AULyETRiD40EkM0UqH70dAjPfQDB8wqK53TSz8rGBqJIfOMRuYZQ89Whb5vn9ldd5GJohF9aCSGaKRC96OhR4L0t7+Z/fa3BgAA4BoYeioV+u7dZvfeSyaKRvShkRiikQrdj4Ye6aHv3Gn2u985p5d+VjA0EkPmGY3MM4aerQp9xw6zmhoyUTSiD43EEI1U6H409EiQ9HpdnQEAALiGLw29p6fHamtrraqqyiorK72tq6urz/2PHDlis2bNsry8PG9Lu0Lfts1s6VIyUTSiD43EEI1U6Nmgvr7empqaIuPGxkar6WMp/MSJEzZ16lTbv39/2hV6pIf+3HNmDz/s3BuDflYwNBJD5hmNzPOwM/Ti4mLr6OiIjNvb262goCDhvjL6l156KaXf32eF/swzZo88QiaKRvShkRiikQo9G8Qvm2sJvq+ldBm9TH3UqFHePhUVFf0uzycy9EiQnnrKbPVqAwAAcA1fGnpOTk5Sr4kRI0ZYS0uLZ/rd3d22Zs0amz17dsJ9W1tbPTPXEn40RUVF3vJIp8z8sce8zErjcIY11OOdO3c6pSfReP369U7r01hxRF9m4/Dm8vvR9c8Ln2c+z+mOh32FLkOP3zc3NzelCj3SQ29uNmtocC4ro58VDI3EkHlGI/M87Cr0kpIS6+zsjIzVQ8/Pz0+4r8w4ut8uxowZk5KhR3roa9ea/fnPzr0x6GcFQyMxZJ7RyDwPO0NvCFXJ0cvi+r66ujoyjl5+18/q6uoiffO9e/fa0gEuPeuzh66/+fjjBgAA4BrD8jr0+H76smXLvJPitNQuc7906VJ6FXp9vdmGDWSiaEQfGokhGqnQ/UCfPfRHHzV78knn9NLPCoZGYsg8o5F5xtAzNPRIhb5ypdmmTWSiaEQfGokhGqnQ/WjokSCtWGG2ebMBAAC4BoaeSoW+bJnZ1q1komhEHxqJIRqp0P1o6JEe+pIlanw4p5d+VjA0EkPmGY3MM4aerQq9ttbsxRfJRNGIPjQSQzRSofvR0CNB0rXuKT7oBQAA4EqAoadSod9/v9muXWSiaEQfGokhGqnQ/WjokR764sVmLS3O6aWfFQyNxJB5RiPzjKFnq0K/+27dO5ZMFI3oQyMxRCMVuh8NPRKkRYvMXn3VAAAAXANDT6VCV7Bef51MFI3oQyMxRGOwK/RmPU/ch4Ye6aHPm2f29787p5d+VjA0EkPmGY3MszOGrqegjR8/PubJaL6q0OfMMXvrLTJRNKIPjcQQjcGu0MeMGeOZurYluuuaTww9EqQ77zQ7csQAAABc44r30Ldu3Wp5eXmeqesZ5a+99pp/KvTbbzd75x0yUTSiD43EEI3BrtCjUT99xIgRkYo9vOk11ww90kMvLTU7dsy5Nwb9rGBoJIbMMxqZZycNvbGxMcbEw9vIkSPdrdBvu83sxAkyUTSiD43EEI1U6Fu2bIksuY8ePdoOHTrkXKD77KGXlJidOmUAAACucUUNfezYsZGqfMWKFWn/np6eHqutrbWqqiqrrKz0tr7OnI9f0g9vaVXoxcVmnZ1komhEHxqJIRqDXaHLSAsKCuz06dMZia6vr7empqbIWMv3NTU1ff7NTCv0SA+9qMgsQ+2DAf2sYGgkhswzGplnZwz92WefzYro4lCl3NHRERm3t7d7icJgGXqkQp861eyjj8hE0Yg+NBJDNAa7Qs8W6sFHoyX4+NeiDV0n2mmbMGGCLV++3C5evJiSoUeCNGWK2T//aQAAAK7hS0NPVHUnU4mrqpehL9JDVhLQ2trqmbmW8KMpKirylkd6tArw8cdeZqVxOMMa6vHOnTud0pNovH79eqf1aaw4oi+zcXhz+f3o+ueFzzOf53THw75Cj0f75ubmplShR3ro+flmn3ziXFZGPysYGokh84xG5nnYVeglJSXWGXW2uXro+TLbJDh37pxdc801KRm610MPJQKW5N+40tDPCoZGYsg8o5F5HnaG3tDQELMsru+rq6sj4+jl97KyMmtpafEqc5m5Lncb6JK5hD30S5fM+jjxDgAAYKjxpaEPdB16tKFv3LjRSktLvbvQjRs3zlauXDng709YoV+4YDZ5MpkoGtGHRmKIRip0v5Cwhx6q7i3cS3cM+lnB0EgMmWc0Ms8YeoaG7lXoZ86YFRaSiaIRfWgkhmikQveroXtB0kl4uvUrAACAg2DoyVboujPdtGlkomhEHxqJIRqp0P1q6F4PXY9N1eNTHYR+VjA0EkPmGY3MM4aejQr9vff0DZkoGtGHRmKIRip0vxq6F6R33zWbOdMAAABcBENPtkJXNlVeTiaKRvShkRiikQrdr4bu9dDfftts9mwn9dLPCoZGYsg8o5F5xtCzUaEfOmRWUUEmikb0oZEYopEK3a+G7gXp4EGzBQsMAADARTD0ZCv0V1816+M56mSiaEQfGokhGqnQfWDoXg993z6zu+5yUi/9rGBoJIbMMxqZZww9GxV6S4vZ4sVkomhEHxqJIRqp0P1q6F6QXnnF7L77DAAAwEUw9GQr9JdfNnvgATJRNKIPjcQQjVTofjV0r4e+fbvZgw86qZd+VjA0EkPmGY3MM4aejQpdrz30EJkoGtGHRmKIRip0vxq6F6Tnnzf7/e8NAADARXxp6D09PVZbW2tVVVVWWVnpbV1dXQP+u82bN1tOTk56Ffpzz5k9/DCZKBrRh0ZiiEYq9GxRX19vTU1NkXFjY6PV1NT0+2/27dtnc+fOTcvQvR76pk1mK1c6+cagnxUMjcSQeUYj8zzsDL24uNg6Ojoi4/b2disoKOhz/6NHj1ppaamdP38+/Qr9qafMVq8mE0Uj+tBIDNFIhZ4t8vLyYsZago9/LYyW4ktKSqyzs9Mbp2PoXpA2bNDSgAEAALiILw09kSn3ZdS33367HTlyZMD9RGtrq2fmWsKPpqioyI7cf799uGxZJLPSckk4wxrq8c6dO53Sk2i8fv16p/VprDiiL7NxeHP5/ej654XPM5/ndMfDvkKXgSfaUqnQvR66evZRfXuXoJ8VDI3EkHlGI/M87Cr06CV0oR56fn5+2tX9QIbu9dD/9CezdeucfGPQzwqGRmLIPKOReR52ht7Q0BCzLK7vq6urkzLttHvo6p+rjw4AAOAgw/I69Gwbulehr1ljtnEjmSga0YdGYohGKnS/kLCHrmvQdS26g9DPCoZGYsg8o5F5xtAzNHSvQtdd4nS3ODJRNKIPjcQQjVTo/jR0L0i6j/vWrQYAAOAiGHqyFbqetBb3OpkoGtGHRmKIRip0Hxm610PXs9D1THQHoZ8VDI3EkHlGI/OMoWejQn/gAbOXXyYTRSP60EgM0UiF7ldD94J0331mr7xiAAAALoKhJ1uhL15s1tJCJopG9KGRGKKRCt2vhu710O+6Sw9Vd1Iv/axgaCSGzDMamWcMPRsV+qJFZq++SiaKRvShkRiikQrdr4buBWnBArODBw0AAMBFMPRkK/SKCrNDh8hE0Yg+NBJDNFKh+9XQvR767Nlmb7/tpF76WcHQSAyZZzQyzxh6Nir08nKlVGSiaEQfGokhGqnQ/WroXpBmzjR7910DAABwEQw92Qpd23vvkYmiEX1oJIZopEL3q6F7PfTbbjM7ccJJvfSzgqGRGDLPaGSeMfRsVOjTppl1dJCJohF9aCSGaKRC96uhe0EqLjbr7DQAAAAX8aWh9/T0WG1trVVVVVllZaW3dXV1Jdx33bp1NmPGDMvNzbW8vDybPXu2HT16NPUKvbDQ7MwZMlE0og+NxBCNVOjZor6+3pqamiLjxsZGq6mpSbhvWVmZtbS0eEmAtubmZpsyZUpKhu710LWdO+fkG4N+VjA0EkPmGY3M87Az9OLiYuuI6me3t7dbQUFB0v9+5MiRqVfokyebXbhAJopG9KGRGKKRCj1baOk8GlXe8a8loru72zZs2OAtwSeitbXVM3NV/PGG/snEidb26Z3iNBHKrsITwpgxY8aMGQ/12JeGnpOTk9Rr8T/Xpn776dOnU6/Q8/OVOZCJohF9aCSGaKRCH+oK/dKlS95JctOnT0/J0Keqfy5DdxT6WcHQSAyZZzQyz8PO0EtKSqwz6hIy9dDzUzBcnfGeiqGXKgG49loyUTSiD43EEI1U6NmkoaEhps+t76urqyPj6OX38vJy279/v1fFX7hwweuhV+hRqCkY+kLtP8CZ8QAAAEPJsLwOPdrQN2/e7F26pjPbx44d613edm6Ay88uq9BvvdVs6lQyUTSiD43EEI1U6H4i3tALr7vOrKjIWb30s4KhkRgyz2hknjH0DA29TPdx161fyUTRiD40EkM0UqH719AXlpeb/epXBgAA4CoYejIVuqpzPT6VTBSN6EMjMUQjFbp/Df2miRPNSkud1Us/KxgaiSHzjEbmGUPP0NBn3nRT6D8zyUTRiD40EkM0UqH72dAX6tavd95pAAAAroKhJ2Hot994o9mcOWSiaEQfGokhGqnQ/Wzot4wfbzZvnrN66WcFQyMxZJ7RyDxj6JlW6Ndfr0iRiaIRfWgkhmikQvezoS/UNeiLFhkAAICrYOhJGPoduvXrb35DJopG9KGRGKKRCt3Phv6ra64xW7zYWb30s4KhkRgyz2hknjH0TCv0ggKz++8nE0Uj+tBIDNFIhe5nQ19YWGgW9bx1AAAA18DQkzD08vx8s9paMlE0og+NxBCNVOh+NvRbx4wxW7LEWb30s4KhkRgyz2hknjH0DA39zv/9X7Nly8hE0Yg+NBJDNFKhZ5Oenh6rra21qqoqq6ys9Laurq6E+zY3N1tpaanl5ubaqFGjvH/T3t6ekqEvnDzZbMUKAwAAcBVfGnp9fb01NTVFxo2NjVZTU5Nw37KyMmtpafGSgO7ubnvsscds2rRpqVXo48aZrVxJJopG9KGRGKKRCj2bFBcXW0dHR2SsirtAl5YlyciRI1My9Bl5eWaPPursG4N+VjA0EkPmGY3M87Az9DwZbBSqvuNf64u9e/d6VXsqhj5LN5aprycTRSP60EgM0UiFnk1ycnKSei2e7du3280339xnD721tdUzcy3hxxj6//yPtd1zT2QC9FXZFWPGjBkzZuzKODAV+sqVK62ioqLPk+f6rdB//nOztWvJRNGIPjQSQzRSoWeTkpIS6+zsjIxVcefr5i99sG/fPu/EuGSJN/TSn/1Mp8s7+8agnxUMjcSQeUYj8zzsDL2hoSFmWVzfV0fdmjV6+X3Xrl1JVeX9Gfrs0aPNnniCTBSN6EMjMUQjFXo2Geg69GhD1/eJtlQMfeEvfmH21FMGAADgKtwpLpkKXf35Z54hE0Uj+tBIDNFIhe5nQ5+piv6555zVSz8rGBqJIfOMRuYZQ8/Q0Ofk5ppt20Ymikb0oZEYopEK3c+GvlAnxcW9BgAA4BIYehKGPnfECLMdO8hE0Yg+NBJDNFKh+9nQy3/0I7OdO53VSz8rGBqJIfOMRuYZQ8+0QtdJcbt3k4miEX1oJIZopEL3s6Ev1J3i/vY3AwAAcBUMPQlDr/jxj3X/WDJRNKIPjcQQjVTofjb0WT/4gdlrrzmrl35WMDQSQ+YZjcwzhp5phf7DH5odPEgmikb0oZEYopEK3c+GvlBL7m++aQAAAK6CoSdh6PO05P7222SiaEQfGokhGqnQ/Wzoc77/fbO2Nmf10s8KhkZiyDyjkXnG0DM09PlXX2129CiZKBrRh0ZiiEYqdD8b+sL//E+z48cNAADAVTD0ZCr0733P7ORJMlE0og+NxBCNVOh+NvR5//ZvZh0dzuqlnxUMjcSQeUYj84yhZ2joC2ToXV1komhEHxqJIRqp0LNJT0+P1dbWWlVVlVVWVnpb1wCG29bWZqP1XPM0DH3hv/+72ZkzBgAA4Cq+NPT6+npramqKjBsbG62mpqbP/XNyciJbWhX6v/6r2fnzZKJoRB8aiSEaqdCzSXFxsXVE9bTb29utoKBgwH+XtqH/y7+YXbzo7BuDflYwNBJD5hmNzPOwM/S8vLyYsZbg41/LpqEv/M53zLq7yUTRiD40EkM0UqFnk0TGnIxZD7RPa2urZ+Zawo83dGVT4QnQV8aMGTNmzNilMRV6shU6mSga0YdG9KGRCj27lJSUWGdnZ2SsHnp+fv6gGbrXQ3cY+lnB0EgMmWc0Ms/DztAbGhpilsX1fXV19YDGndFZ7mSiaEQfGtGHRir07DLQdejxxh192Voyl69dtuSuG8sAAAA4DHeKS6ZCd9zQyZbJ6NHIPKORecbQkzD0eY4bOv2sYGgkhswzGplnDD1DQ/eetkYmikb0oRF9aKRC97ehL7z6agMAAHAZDD2ZCv0//oNMFI3oQyP60EiF7ndDn/P97zutl35WMDQSQ+YZjcwzhp6hoc/7wQ/IRNGIPjSiD41U6H439IU//rEBAAC4DIaehKFX/Nd/kYmiEX1oRB8aqdD9buizHF9yp58VDI3EkHlGI/OMoWdaoTu+5E62TEaPRuYZjcwzhp6EoS/82c8MAADAZTD0JAx97k9/SiaKRvShEX1opEL3u6GX//CHTuulnxUMjcSQeUYj84yhZ1qhjxhBJopG9KERfWikQve7oS/87/82AAAAl8HQkzD0Obm5ZKJoRB8a0YdGKnS/G/pMx0+Ko58VDI3EkHlGI/OMoWdaoeflkYmiEX1oRB8aqdCzTU9Pj9XW1lpVVZVVVlZ6W1dXV8b79mXoC6+5xgAAAFzGl4ZeX19vTU1NkXFjY6PV1NRkvG9fhj7b8ZPiyJbJ6NHIPKORefaloRcXF1tHR0dk3N7ebgUFBRnv25ehlzp+pzj6WcHQSAyZZzQyz8PO0PPietpaVs/ro8+dyr59Gfqsn/+cTBSN6EMj+tBIhZ5tcnJyknot1X1bW1s9M9+4cWPM63fccYcXKDY2NjY2Nle30tJSKnQAAICg4YShl5SUWGdnZ2Ssvnh+fn7G+wIAAGDoV5CGhgbvbPUw+r66ujoyjl5SH2hfAAAADH2IGOja8mhDT+c69HieeOIJr7fu6qYkxWV9aCSGaCSGaBx8ffv27fOfoV9p4s96Rx8a0YdGYohGv+sLpKGnmvWgD43EEI3EEI2u6wukoQMAAAw3MHQAAAAMHQAAADB0AAAAwNBTIRuXuw0WbW1tNnr0aOf0Njc3e7cezM3NtVGjRnladCMfl+K5bt06mzFjhqdRdwycPXu2HT161Nk537x5c9Yvw8wU6Um0uRbDI0eO2KxZs7x5jr47pCsaXY+jPrvz58+PxK+8vNyOHz/uVAy7u7utrq7OO97omLh06VJn5jnV4/RQ6A2Moafz2NUrfRBwTW9ZWZm1tLR4b0x90B577DGbNm2aU/GM1qhNSciUKVOcnHOdBTt37tyYuXZBY1/PQnAphidOnLCpU6fa/v37ffPZ1lMhpdkVjYWFhd49OMKfFSXDN9xwg1MxXLZsme3du9f7/vTp07ZkyRJbs2bNkGtM5zg9FHoDY+jpPHZ1KA+qruodOXKk8/F0UaNWDbTacf78+Zi5dkFjf4buSgx1IHzppZd89dnWHSy3bNnijEatYsnIXf6sxFfAqmjDCbqLn5X+NA2F3sAYuusPdYl/o7ioV5mzKmJX9WkVYcOGDd4SvEsadVCKfgZB9Fy7oFF6dGDXNmHCBFu+fLldvHjRqRjqQChT11Ks/n5FRUVk+dLF96KWsouKipyaZ61eqdrdtGmTnTx50rZt22b33HOPUzEcMWLEZUmHEhGXPivJHqeHQm9gDD2Vx666oM81vdu3b7ebb7450kN3TV94OUx9Ki3VuaTx9ttv9/q/iTS4FkdVFDL0RYsWOaVPB/ro9o+MSedLuPrZXrx4sfeZcWmeZeLTp0/3TFyJm/rpZ86ccSqGmlMtU2uONdd79uyJGLoLGlM5Tg+FXip0KvQBWblyZUxF5Go8L1265PUFddBySWN/J0u5GEdpcKkqChu66xrD6OSp8LkmLn1etEp0+PDhyFjGGX7etkurWUo0wifiqm0xefJkKnQM/fI3s8uPXY1/o7iiVydyqTLyUzzDB3pXNUbPtYsaz507Z9dcc41T+nTyVnQ/UowZM8bJGMqQwid2uTTP4X55otdc/azs2rXLO1PcFY2pHKeHQm9gDN31x67Gv1Fc0KsPU1+XWbgST116ozOflf1euHDB66FrNcHlOXftccDRVwrIzHUAXbFihVMx1N/V5Uzh96MMM3xJk0vzrAo4fA6Ha58XfVa07B6uFsMrb65+Vg4cOOCdmR9u87mgMZXj9FDo5Tp0Bw7url636ofrk3VdtwxJlcbYsWO9E6dkSq7PuUvvy40bN3pLr1rWHjdunHegd/Fzo0uatAyrFRiZu1osrmnUe/HgwYNOHn/09/R3FT9t6qWHe+guxTCsT7GMPvdkKDWmcxzkOnQAAADA0AEAADB0AAAAwNABAAAAQwcAAAAMHQAAAEMHAAAADB0AAAAwdAAAAMDQAeDKsnXrVu9e77pbWyJ0tz79XPsBAIYOAA6j27Dq9pdr166NeV33r9bregQrAGDoAOADdG9qmffLL7/sjbdt2xZ5Jj0AYOgA4BP0cBQ9e17PeNZT7vT11ltvtYsXLxIcAAwdAPzE6dOnraCgwKvMJ02a5MTT7AAwdACAFJGBY+gAGDoA+JjwkrueXa2T41hyB8DQAcCHLFiwwKvMX3jhBW+8fft2b6zXAQBDBwAfEL5sTZepRaNKXa/r5wCAoQOAw4RvLKMbyCRCN5zhxjIAGDoAAABg6AAAABg6AAAAYOgAAACAoQMAAACGDgAAgKEDAAAAhg4AAAAYOgAAAGDoAAAAgKEDAABg6AAAAIChA8BnH8arrvLtBgAYOgBEGTq6AQBDB8DQ0Q2AoQMAxoihA2DoAIChAwCGDgAYOgBg6AAYeprk5ORg6AAYOgAExdD72w9DB8DQAQBDBwAMHQBDT9aMm5ubbdKkSZaXl2cVFRV25syZGKPu7u62uro6GzVqlLfV1tZ6r4X3id4wdAAMHQCSNfTKyvS3BIbe0NBg586ds56eHs/cq6urYwx9+fLltnbtWu/n2jZs2GDLli2jQgfA0AEgI0M/cCD9LYGhR6PKe/To0TE/Gz9+fKQiF5cuXbJx48Zh6AAYOgBkZOhZJJEZjxw5MuZn/e2DoQNg6ADgoKGrEv/FL34R8zNV4/EVengfDB0AQwcARwz9yJEj3vfqjz/99NO2ZMmSGKNWD72xsTHSQ1c/PbqHPnbsWDt69CiGDoChA8BQGnpZWZnl5uZ6Z7DrbPaLFy/GGLqqc53Zrn206XtV6WE2bdoUOQMeQwfA0AFgiAzdj7oBAEMHwNCjUMWNoQNg6ADgc0NHNwCGDgAYI4YOgKEDAIYOABg6QAAN3a8bAGDoAAAAkAX+HzMh+Q8YnL5DAAAAAElFTkSuQmCC",
      "text/plain": [
       "BufferedImage@1a32aa9c: type = 2 DirectColorModel: rmask=ff0000 gmask=ff00 bmask=ff amask=ff000000 IntegerInterleavedRaster: width = 500 height = 400 #Bands = 4 xOff = 0 yOff = 0 dataOffset[0] 0"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "val rocs = roc2(preds, tcats, 1-tcats, 100)   // Compute ROC curves for all categories\n",
    "plot(rocs(?,6))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": []
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfQAAAGQCAYAAABYs5LGAABNHklEQVR42u29aXBVV3b231WpVCqVSqXyJR9SqVQqX1JJp9PJ20mnO+m3Q9zmxW3/XZ24C4EYZQYZxChAzGCBMDa2sWRhsLExg5FlIcxksAwGg81sEBKyMZMNls1gkITEbARC67+fc88RV7IEmkDn6v6ewy7dfe+5l+esfc559lpr731+ZAAAAACIefwIEwAAAAAIOgAAAAAQdAAAAAAg6AAAAABA0AEAAAAEHQAQuYh+9KNml8b278zYvHmzpaWl2W9/+1v7sz/7M/ujP/oj+8M//EPv9f/+7//awoUL7fbt25xEACDoACDosW6bX/ziF3blyhVOJAAQdABiS9CxzQ/L2LFjOZEAQNABCJdogTv467/+a0tPT7edO3farVu3vPeOHTtmjz/+eD2b/cVf/AXGAgBBByC2BL2p/SV4Er+/+qu/8vLMf/zHf2xdunTxXjf1+0391t04NfxMOezZs2fb3/zN39gf/MEf1O33+eef26BBgzxRDvj8+Mc/tqlTp7Y5RH7p0qV6HPT7AAAEHYBOIejdunVrcdi+PQT9V7/61Q/2Xbp0qSfuTXGQsFdWVrbaZuq8RP/e3/3d33EiAYCgAxAuQW/NoLicnJwfiGxFRYWVlZXdd0FvWIqLi+vVJ0yY4Hnx8+bN+8H7rcWiRYvq/dbMmTM5kQBA0AGIfUFv6J1ruldrxLk138nNza0XQu/Tp0+9z6urq733JerR7//t3/5tq+ylHPqf//mf1/sdRrkDgKAD0CkEXfnphnntByXoDREttncrrcl7f/nll/aXf/mXdb+h/+uLL77gJAIAQQcgfILemv1bK8D3Q9Dvljtvy4h+jXTXaPboke0K7wMAEHQAOo2gN/TQO1LQ//RP/7Te58F0s7bg1VdfrddR+OlPf2rffvstJw8ACDoAnUvQf/7zn9d7T3nm5vx+wylt8oILCwvbJOi///3v633+1ltv/WCfa9eueVPq7gVNT+vZs2e93xs6dGhdXh4AgKAD0KkEXfPPo9/T/G/l0T/44IO7/v7f//3ft9tUtwD79u37waIveXl5nqcuTlu3bvWmrTXnWDW3nZX0AEDQAYgbQdecbi3gci/Bi174RdB0r4b7aNR4WwRd0DQ6PUilrQLcEjFH0AFA0AGIeUEXTp8+bf369bM/+ZM/8ULpv/71r2379u319lWuvSFSU1O99yXAAwYM8DoHbRV0obS01HtSmtIB+n11JvR//OQnP7Hk5GTPk0fQAUDQAQDNgKZ0RQtec/LWAAAEHQDQgdAa7vPnz/dWhxOOHDnieenRgq5QOAAAIOgAhPlivEc4OikpCSMBABB0AMKOXr162c9+9jNvHrhy1SpaVU3TvtavX4+BAAAIOgAAAICgAwAAAABBBwAAAACCDgAAAAAEHQAAAEDQAQAAAICgAwAAAABBBwAAAACCDgAAACDoAAAAAEDQw4bi4uJ69cGDB9vTTz8d2jJq1KhQ84MjNoQjNoTj/ec3evRoBL0hPv7443r1/v37h5rvtm3bQm9TOGJDOGJDON5ffhJ1BP0egq6eVJjx9ddfh96mcMSGcMSGcLy//BD0Zgh6S40EAAAAPGgg6HjocMRzgyP84IiHHh+CTg4djvCDI/zgGHZ+CDoeOhzx3OAIPzjioceHoJNDBwAAEHYg6HjocMRzgyP84IiHHh+CTg4djvCDI/zgGHZ+MS3opaWl9uijj951n9raWsvOzraMjAxLT0/3SlVVFR46HOnRwxF+cMRDDwO6dOlSV+6GnJwcy8/Pr6vn5eVZVlZWiwSdHDoAAICwI+ZD7vcS9OTkZKuoqKirl5eXW2JiIh46HOnRwxF+cMRDjyVB79atW726QvAN3wugh7JIzOXFR6NHjx5eviNoAP0NU33NmjWh5qe/S5YsCTU/1WVH+LWt3vBvvF4vH2/72Eq/LDW7bnZq3ynbVbDLvi351sz5Fmc/Pmt73ttjp4pOmZVF6nvX7rXT+0+buX+75u+ywhWFdmbnGbOv3C5ryqx4ebGd/eSs2XFXX1tmJctK7Lut35kdck7KinLb/9Z+K95TbAe/PmiLji6ylw++bCuOrrCCMwW28PhCm/v5XHvz5CJbejHTZn4/yCbf7GOZ14dYdvUwm1mdZBNVv+bqN4bZrBsDLO1Wr7r6c27/sa6edXWoZV8bZi9cHWypql9y9SvD7MUryTayJrGunun+Dq/pWVfP8uuZUfWRrj738lOWfdnVLw71fu959ztBfezNXvbc1UF19fE3e9vsawPr6pNu9LGM60/W1afc6GszrifV1ad938+mXe9XV0+/3t+mXBpsL5ROtVmfzbYxWxbakHfzbNqWJTb5g6U2amW+DVy00aate9sWvDnPtk2daqeeHGLFaeOsaNw4Kxk51k73G2pFYybYgTHjrXhUmn3XZ4gdGD3JCkdMsf3Dptn5xKG2b+jTdeVcz6G2P6p+tmeKFT41rd4+35R81q7nX6cX9MY+v9d38NDhCL925Fjjyg1XrrriNMwuRhXVFUA7Z56Y2afmiZYdc+ULV953Zbcru1zZ5spCV1a5ssGV91yZ58oyV1a48rYrs13JdOVNV95wZab/3nx/3+lm36d9bzbXvX7BlcmujPX3c5+ZHlaV7Eqa/36KK0n++yrDXOnnygj/tfbt5cpAV/r7r7u7kuC/7iuPwJVB/r5DXOnpylD/t1US/b8jI//H7b63I//XeFcmRr5bOaPS9uXss9xtuZZyIcWGVwy3lKoUe9KJWkJtgvW51dvGXEyxWWVjbMalZHuxcpgtPj3JK3OqhtiLVwfYSieKGy4n26pvJ9q7x2ba9gNveGXD4ect/6unbXvxG14p+OJFe+fktLr6ps+zLOfrSbb9c1c/+oZtOTHPlpxNs+1fv+GVj75eYK+XTrctR5fah4fesoKS5TZ3X6atL8yzdZ+utDV7VllGQY69s22d5XxYYEvf/9DSFuywBas+sjfWbvPKhFe3W/aKbfbqyu02f8VOGzu30J59c49l50RK2kuFNmPBfnt2QYnNeOmIDU49aylpZ2z0pLM2Ytx56zvgivUbeMWSh1V5RfU+SdcsaeA169232hJ61Lpy254cfNVGjjpnmwYutNJeY6wkeZZ99lSGHR0w2c73SrHqnn3sbNJI+3To87Zx4HxbOzXX1k1729ZPWmKbn8qyDenLI2XaEvtocKYVzHLH/Nxi2/L867ZzVIZty1xYV3a4+taXXrNtWQu9smP0LNv64oK6+ua58+3C6dN46PfLQ29K0Mmhg2aj1pWbrlxy5RtXKv1ywRXnsMkjs7P+ZxKpr3wBO+rKRlc+d+UzVw7KlXRlryt7XNnhi1SBKx/6+77qykpX1ruyzpWXfXFb6Qvci76QLffff9Z/b7Ff5kQJ4DNmVyZdsYoJFXZl2hW7MuWKVaVVWcVYV594xSsXx1+08jHldiXN1ce5eqqrj3L1Ua4+4opdGnbJyoaX2ZUhrp58xS4Pumznh523K4OuRN5LuWLfjfrOyiaUWWl6qR197qityltlB+YdsKIFRVa0sMjWrFhj+5bss6Jlrp5TZOtWrbO9K/da0RpXX19kGwo22O6Nu6xos6t/VGQFTiB2fPKRFRVutqJDa23jN8/a9tKXrejrZVZUusw2n37Otp2aa0XfuvqpZfbR2efto3PPWlFFlhVdnGXbvh9mm6sHWdH3Y63oxljbfnOobbw1wIqqx3pl580U21DzpHt/tH14a5Tl3kq252t7W37NMK8sv51sz9b2svzbrl47zHJrh9gsp9D5Tu1VVjilnuEUfKlT8tecor/klH+QU/tZ9qQzeaQkO8Wf6XoDQX2Iq09yPYFxrgy33u7XurtPe7imetL9ZoptsQHulHnKnUoD7Ir77QsXh1ll5US7cmWuK4vswoXXraIix723wcrK3rcTJ1bakSOb7csvL9sh59Fv3aoohesPuQ7R0qVmM2bIaYn8ne46Namprg/hOhFjXYdGT+N86imzPn1cnyMlUlTv5TorAwaY9e7t+i4JkdKvX+Q9fben67QMHx75Xf2Ovj9litmsWWbPP+/6Lq7z8tJLZtnZkaL/c67rZL32muuHvaF7rtmCBRF+KjNnRj5bu9ad/u78X7VK92nzjue4u36+cJ2/L790fcLTkXLUXU9fuWur0l17166Z3bql69JdmBvdhTN0aOQ/ePdds507I2XTJncNrYvsHMPo9IKe4s7ASrWqD+XQE3T24aHHD0d5iN+JhO/5HXBF4yTzfDFcFRHDy7Mvm73lXi/1vbcMV9xNxdwNx92Vzca4MtX31Jw3VtOrxhOqstFldnz6cduZtdOOTz7uvf5i9he2/ZXtdvzZ416ReH08/2M7/ryrzz1ux14+Ztte32Ylb5bY3rf32rZV25xXM9fe/+B9K9hYYAWbCixrX5a9t+U9K9jq6h8X2Mv7XrY1u9ZYwe4Cryw4sMDe3fuuFexz9f0FtvDgQltxYIUVFLl6cYEt+nyRLTu0zN459o4tPrnYpn833aaVTbOM8gybXDXZBl8fbEnVSda/xslFbQ9LrE203q48ebufPVnbz/o6oepV29N7rdKvto/bp369pxOaoN6/tq+rJTjZ6eeVJFfq1/u6WoKTpMHO+R3uTDnWiVs/J2JpTqzGuDLCyVSiM3uyez3EK0NdPd19e7b79mwnbsOcuE13vzHbLyNcfaondn08wRvt3OJJ3r5PeWWMe3+C+z+CukRy7G3tO9KVsTb+9nBLvZZqMy89Y+llz9uYL+fakKJsm/rlApt8ZKGN3rfIBm1ebjM+W27PfZZns/a9a2M3rrZXPlvnlbkH1lhqwZq6elbxWhu14U59Xsl7NsIJxfw9H9krH+22eesP2rD0bywz76hl+WXEzFJ7MedOffiMr2zOmycsc9Epe3H+ORuResnS0y87wb1qkydfd6J50wYPvmVJSbXWo0dEPPv2NXvyyUiRyKqenBwRYImsXk903n56utmECeZ+x53qb0VEXVomkfzcdSYPH3b9Sdeh3LdPs4juCKTEsawsUr5z19L27WfssrtcbtyQk9TgetMPDRsWUeygiIgUO6hL7aXsQV0K/9xzd+pS+GeeuVNXryAj4059/PjIwQT1SZPMpk27U3e9hxvjxt2p6zNNP54zJ3Iw5NBjR9Cj38vNza2XE9frzMzMFgk689DbgGpXqszLCVq5Rcp5V760O/UyX2i/9d//3BdbNcMHvuAqHPq675W+7Ic8M3yv8xk/ZDnRF1z9dTcxpxteePRS8iVb9/Y6W7N6ja1/f73lb8+3OUfn2KyTs2zGdzNs1OVRllgjP6hlm8QpyW0ptSk23AnDoNpBNtnf0tw2wG1BfYLbtO/kqM1JojuEDOc0v+gc6fk23m2L3LbY3ya57XW3BfXhF4e7w3+1rj7dba+4LajPcNvLbgvqs9ymz991m5N45y2u8P4etELn4W2zo27fb93/f9X9P7e9OLLixu5Ga9P8MsY3bFAf58eMg/p438hBfaJzgvpF1RXPfjKqPtWPS0fqNTXpzpNMdcLwiut05zqxKHCe1nt25swuJxp7vHL48PvuPdfp2XvWPvjgivPwbtur7hxYtMhs4ULdx2s9HXjllUiRFyhvTlqh+790ZcSIyP1eRfWkpBueviQlRTxLCeLAgRHNGeIOb/DgyL5Tp0bET9/Rb6pICOXgBXXtI7EM6rqfykMN6vJ6JajSMvFetizyvibeyEFUmT1b96U79bS0cu9zOZPyplesMNuxI6KT8kYPHnSXirtWrl41u307RPeckpKIiy9xVgPt2mW2e3ekqPcQXc/JiRxUUH/nnYjLHdR10B99dKe+cqXZ5s136qtXRwwU1OVdv//+nfr69faVZjMF9Q8+iPzt5PftTjFtreH0tejXzENvRog4yG9eiYivJ6wKAx9xZb8vqIt9rzYI3epe/pyfexzr5whT/bziAD9v2MPPI7obZ21ibSRvqPKUnzcM6kFeUd+f5HvDT/nC7e4LNe/UWNnrZXZ+83n7du+3dvTgUXvv4Hv26dFPrejLIis6UWQbjmywj7/+2LZ+t9XeL3/f5lbNtdnfz/bEVkI62m3PuW2p2/Ld9obbJHSfue202667rSV2rLXaB9S63/vhhSN28eKrfiJ5l1/cTdC2RNXVQGv9v6/7guwUyQl7pKT6xu7jJ4lnRQyskVXeCdA6SFSuuHPnzBmzDRvOe/d1RTF1j1WIVUIm0ZLDJI9SoVmJqIoENfAgJajyLgMRVlGoVs7Yiy9GNEJ98UDsdH+XTmzYENECFemC7uvyMMVj714Ndo2EY1UOHJA3+p3naUoQa2s72fX8IDnW1ERi2zK6ejHyqiXSHdXLiDE74qE/IIQ+h67r5bpFBhN969/jNZjoEz/H+pyfO832c6TJvtAO94Wyly+oEtGEKOF90ne4knyBnur/lrsp1z5bG8nLrvc1QyJfqF65K+6GWbOvxq6cuWJlFWV2/OJx2351uxXXFttut8krnOe2An9b57ZMt0lQ33Lba24b7jYJ7tNuk6ea6LZBbuvttp5eYLeHo64Q6hib4rYhbnPdM3d4kW2Y2+SRLnCbvNMX3Pae2866rf3FVyJ70k9wf+WXw36yO6gfjUqSB4ny7a7s85Pgeb6oLooqY/1Y/1S/sdQ4vf3Xem+kn/jO9MuYqCR4pu9Bz/UFXf+HRpTt9BPyKh/5f2u8e7EE+MSJSEhV4VV5fhLiPXsiIVjlMp99NpLLlKAqZCvvdcyYwNO1upCvXiu3qmCWPFA5R/q+BH3LloiQnnQm++Ybd+pej2gBeMBQjFw9HYUv1Et64YVIUThava2grlC2xDmoq0Hl1AR1hcIVplDPTL+zZEkkxBDG3lGcAUEPk4d+07/vv2aRQU4atbvQ92Kf9u/fg/z7fB9fnN11V5Pk7o7pvoDLk3bXqzcwSvpR5OvOlxFHr+ZCjX198Wv7/Mbntvf2Xk9gJX4SU4nsbC+TOcITTImkwsYS1OaEn590W4rb9P3BbnvGbS+5TWHgcTfG1YWCFUZWODnPbRL3TW57022FbvvCbSfd9rnbqtxW7cXs7yekLBqhtsLd7573Pdvn/Zj+IN/oY/xGSPINHxh/sB/fn+jvMzCqPtbvIQX1ND/cPMdv1DxfhDdGlZf9TsFRr7Fu377s5SgvXYrkLjdtOueJrpwg5T517WpAkQYLKdyse6vCzLpfS4gV+UxLi9yvFSqW4EqQJbpB3lWhZg180v1caU0Viffrr0dC2vpthX3l6UoLJPYK954/b3bhgtnFizHqXXYGfjopdK9S46qR1fAqanS9F9TVE1MuQQ2vUMny5ZEwxqefRop6XDqhgroaXKPPgvonn0RGoQX1Xbus8s03Iz0z2hkPPdYE/b7l0K/6EdM0v/TxnSz1HzQyebXvaOn1Z743rhvorfo/E+RiFDZW+FgiLZFc7TblZEf7W7Lb5Pn2dZtE93m36XN5yMqv7nJbkds+dZtC0efddtlttxr+h60IP3dMnl+hDOUQdOEoGb/BDy9o1Nsk3/OV0Ud7PaZz56a4v5v8cEeR73V/65djvtB+X7874PoDWrdIA4kU+i0qihQJoEYS696o0LOcGImmBFJFXq+cIHmyQb5Xp5ne071Xzk/37pEQdJDf7dWrxsvt6jvKxUrE9X/ofqxTVmlHhaElvPKI9VoCfORIJPeq0LPGA93PUDNrfLczP4VPdFLIa9ZgARU5GOqZKU+hHMTbb0dONDW8isRaIZagrvyxTsh2DIXTzuHkh6A/aA/9sO+IKYKq8UPP+vWvfA/dlNK+4STkWy9crbC0wtHPui0IR/dymzzh4f7Wq7aXk6U+3iArCfdQt0msc9220W3r3Vbitkq33baOyW813Rut9UV3n+8pB0WiWhpV3+8Lc1Av9MPHO32hVjJfc7Jm+b0i5RCCCcDjfNHWZzlRwr7PeeXVXhhYorh0aaU34lciKc9XXq48VeWAFWXUACmFkIOBUZq6o0UH5fkGYWg5QCoaDCURnj8/kufVYCf9jv6foEiQ5W0H+V69Jy7qIMj5aSi6eL9x5LnpRNDobOWmdRJqQFdhYaSsX+/6l8ewIR46gt4aQW9TDl15bqUyFUZ3N/7q1GpvqtLBEwdta81WT3Q1AnqO2/R3kDdLNcETZoXC9bnC0fvcFoSj9Ve54XP+dtRtV7wRbWGBlOiyRUbbBeWCH2bQaiBz/d6MQtZK5GsUnXLFY6OK6mOi6lqhIzWqru9rFHWW73Fn+4Yu9oT/7Nly55ne9hwTec7ygIOi0HQQdtY0H4mxZsVoxLLunXJ4Fi+OFDk6GmyriKSijgp5y9vVtSjnibQhaJ9LpjYyyEAnq0I0EnKddMq5AICgd5CHLgFf6Qu4058LIy/YyvUrbenXS23qzaneIC950zPdpilFOW5TjlmC/ZXb5EXXWMtGDD24nqhuLud8b1mj4xSe1ki5F+zOEHXlmhN97zjIN/f260r0L3Flq+9Za32A5h2r7ne6t2lJgVOnFK6KRBGDkLbG7cijDsRanrNGSSucLWHWAC05NhJk3Tclxg1TgKwUB8d2gxYx0aonyoMoPK4QT1Ak1gr1BPVgYINC665HWaWTVQPYaGc8dAS9/QW92Tn0k76ejXbX45bLtqJqhTdQTPlqDf6SZ/19gxxse6B9cjEKxR/1BXqI7xEP9ouOv5cvzMOiPGUNIAumSinhrxF9mlR+854cNc3p3LlIflcjq5UKVKhbg2UVopbXrDUglDOWw6LpTconK22oQVya6qSwdhDSVohcU5mCHHHH2fH+gbxliDiqN6jBEhJfnaASaBXlujXoQSMOlZvRfGvlW7SvBjGoKNQTXZcnHvUAKdo5PjiSQw+zh/6+8x7HVtqeo3tsrtsk5Avdds7zaMPa09NNRFOZ0nwB11/lmIv8z6r8omO40Kr/QaOgFbZOS/u+braMwtwSaI2oVphbo6o1MFehbt37dH+TQGtAl8Lb5eVm1dUWYjvicXR6jsGUr3nzIoKtUYvqUaoHqhNWoR8VCbRCQW0IldPOXM8I+n0U9KaMpGlV867NsyGXhli/G/1s0O1BXh5c86MrrbIDmAeLkCikrUnpGtmt6VEava3Rd/P8MtnurACjHLTmyH3b5v9dnrHWmVAIXPc6RRAl3gqFa3yPZrxo3rOmYQHQaij/opthQUEkXKNpWcHSbMqzqNcY1NUZ1/SCoK6TUiWo6zPtE9T1Xf1GUNdvK0SuKJ3C41rVRoMqmEgPQggEvYUeuqZoaRBaltuSapPslb2v2PaV2+1qzdUHzPKG70kvdve3IByufLWmYmklGa2LqsVKnMvrLVD+vi/yn/giv8+shQuu6AEHWoxEeWs9MEHTWRUq1z1V4XDd8xRlDJZPVq46GDRGjz6OvaLvv494rwo/q4ensHRQVNdNKKjL+9Wc6qCuHIvWcw3qmlIg71gnnUYwKk+jnqLmDaoEy4UGdV3LyukE9e3brUwh76Cuz7RPUNd39RtBXQM1HvDqZ1wrXM8I+n0U9OgcuhZf0VKiBTcK7Pr065HFX+4rrltktZj3fGHWaG4tYKIQuQaZrXX3yuX+fm2DxvEoB62c9MsvR6ZZBU9G0kpgmqal1xJt3Vu19oTuh1pjWiuN3Q3k3ELET70yzZFTvkPCGIilBi6ogYN6sNpMUJf3qrl4QV0DHCSwQd2dGNW6VoK6fl+CrXyybjRBeFonjAaMqWjOdHRdc6Y1hy+oax6hwthBXZPr9XQQ2hmOtDOC3lYPXYPblt5cGlnLPOd+MZBb+5kv3lqdbFiUqAfru15uU09PnrNW/JIzoucpaA611s3WPVzzqLX4ie75wSIlymW3xUmhR/+A+AXTnySUCi1LpNXAKgolS3AlypoErx6ZPNJALDVowXmwdXV5vsEzKlV0Mig8E9SD1WuCuhPeCq3rHdQl1BLkmzdpZ64VOOKhh0PQo4209tJay9maE3GW290T12IqegCHpn1N+IFwtwUao6N7rB5aFIi3nDGN8dE9eP/+0N134wcSYYmlvFmFlxVyVlE9WNZTRblc5XSDusLR8pqDerDEXBCO1uAsLUSiZ2KqKJyiJ5QAADolEPQWeuir96y23K25rfxlDaTR/G3NbZvtF61wptXMtFCKVjLTKmYl7dLTUxhcTxnUNFeFyxUBVRpTXrlGjtNb7mCOEnJ5yFrVRkKtfLCEV3P5VALv169XqDemvG7wuUJ2yk0Hda25ffYsXhGeGxzx0BH0pgQ9Ooe+8uhKW1GyooW/+L0v1JrfrWeDuhu3N6BN5YBF5nG33j1+552iuvE/mvYlp03Om9KfWlRFUdSOfo5CXOfcNCJaK4BpIII8cBXlnzWSUK81JxkbwhEb0s4I+oP10POO5tm7Je8285e00owWalcIPdOvtx0Kj2sRFT2kQ1HWxMRaTx8UPte4Iz0IRA4bveUO5qgchhpKC79rzVl53FqDOygNzjNsCEdsSDsj6PdZ0KONlHs011aXrL5bM1lkAJtWUtNgNnnzX7SZkwakaQaNxjYpfK5R6OrgnTtnoK3Qs0BlWAmvQt8qCm+otxTUNeBAYY+grjx1kNNW0eAzTbkK6gqjywPX07E0dQAAAO4zEPQWeujLjy63dSXroj7V/PNPfBF3N31vsRYNgddj1dr+5A4twjJ3bmRgsnLgSrk2XNOC3nIroXyz1o7VOrJLl9p38qAlvirKTWtAWVBXTkMD14K6RnCrhxXUlfvWKPGgrsFn2h+vCI7YEI546E2jtrbWsrOzLSMjw9LT071SVVXV6L41Tv3mzZtnjzzyiD366KO2QA+ibqGgR+fQlx5dahtKNvg1ucdayEWD3Nb63nn7PH5LK6qJqtKucv6kF02BfFYLoae7aKCBRoRr0MG1azFhR9o5PjhiQ9o5rgQ9JyfH8rUEo4+8vDzLUkK5Ebz22mtW5A86uuTc3fnOI1u2bFmrPfTFxxZbQYmmkzlX2RudXtCuxyYPXCKuqWWaJtychy7RW24mNOxf4Q6FyrVwfINOIJ4bHOEHRzz0B4zk5GSriHo6UXl5uSXqeZmNQF55NOTJ99XKVS0Q9GgjvfHVfNtUqbXRR1h7DXITtDiWphxrYRdNNeuUz9kuK4tMrdKTV9QGOliVIN8c1FU0UV7vB3Xtr1BFUFfURHnsoK5ekPLcQV35bOXBo+ua163R5jxjGgDQCRGTgt6tW7d6dYXgG74X4KGHHvI+j0bXrl1b7aG/dnm4bbkhz7x95oFpsa3gkaDyzlsj5KHqieoJLQsXRgRXgqzi7FcrwdaEeC3yrodqKP8sj1lF/KPrKspH6/2grlXLtAJaUNdQf+Wqg7ry3VrOLqhruVCNJI+uN5GWwXODI/zgiIfeQejSpUuz3hMmTpzoheeVS5ew73dC0JSgF7ubvsRcIfxo9OjRw8t3qAEWlE2yjWefrqsHDdPS+qpV+2z69GveYLesrMv20Ucft/r31qxZ02Y+ba3vdCJ9UUn/AQPsypw59tmyZXZGovzNN3Zm1y7b9MILHcqvOXXZEX5tqzf8G0a+YbheYpmf/i5ZsoTrOYT8Or2HrhD7tGnTPBHXwLjMzEzro3BuKz30eeXj7ZPyZ1vNXc+V0FoiSuNquW09wSyme6IKKbz5ZmQu3aJFTS5BR48erwiO2BCOeOg/QIpzaysr7zxvXDn0hISEZn1379693gj5lgh6tJGyKsbazrIXWsVbEV9FoDX9rKNXbmsXaMSeHnauVW1YIxwAADoUMSnoubm59cLiei3PO0BT4fdDhw7ZgAEDvA5Aaz30ly6Msd1lc1vMWWuX6GFXehRzp+iJao1xLbyi0eINJ8bTo8crgiM2hCMeenNwr3noDQVd4XaV1NRUO3ny3iPT7zYP/YXKUfbp+ZdbxFePKZX26UmS9wPtMp+xujqyWMqKFZGehxZb0V8V5Qc0QjyoaxSfRp2XlDxYjvcZzE+GI/zgGMv8WCmuhR76c1UjrPD8/Gb/lgZna4C3nlzZoT09jT7XSPC33oqMNpdIa1qXisQ6eBxbbu6dEeZK+KuoE6QR5UFdC7Ool0KPHn5wxIZwxEOPJUGPNtLsi8OsqOzVZv3OxYuR5cHffbeNhJozl00RCs1708A0RRTkRWt1GhV50xoIqAeEiIymfJ0+HVkIXkX573tM6QIAABBuIOgt9NAzLg21kvNv3PM39KAtLRTTZjGX8Mp7Vs9Ac9xUtIiOYvh+vVafS8Sff95s/frInGt50krcq2h+tr+8Kb1levRwpJ3hiIcet4IenUNPv/yUHTq/5J4Otcbo3WMw/b2hsLY8ba1uplXWgqIFU+RZ+/UDGiDYHvPf7iPIuWFDOGJDON5ffgh6Cz306VeS7XDZW3f9vsaVKR0tL73V0OpmEvNmJN/pLdOjhyPtDEfaGUFvhqBHG2nK1UF27Pw7TX5Xi/8oZa3HnrYaetSaBq1pOVQAAAAAQW9/D33itQH2VVl+k9+dMcPsgw/a8J8rzN6zZ4vEnN4yPXo40s5wpJ0R9GYIenQOffz1J+1k+eomtVhR8mass9I4lHxPTzfLz2/R18hnxQdHbEg7w5F2RtDb0UMf+31/+6bsvUa/9+qrbVwJTou7T5vW4keu0VumRw9H2hmOtDOC3gxBjzbS6Bv97HRFwQ++o5y5Hsmt5c1bhTNnIquzafQ6AAAAgKDfXw99ZHVfO1vx4Q++s3Jl5DHgrcLt25EHnHz4Ib1lOGJD2hkb0s4I+v0S9Ogc+rCbve38ha31PtcUcOXOtSJqq6DVZ2bNajVf8lnxwREb0s5wpJ0R9Hb00Ifc6mUVlTsa7N9CPf7+e7Mvvoi49dOnm/XoYXbhAr1lOGJDOGJD2hlBv5+CHm2kZCfolRf31Ps8Lc3s4MFm/LAGu+XkRARc68LqdVGRHuhuAAAAAIL+AD30gTU97dLF/XV1zTnXw8vuOTBdT2qRG68l5DQAjt4yHLEhHLEh7YygP1hBj86hJ93uYVcv33kOeFaW2eLF9/hBPSxFSXatua4BcO0M8lnxwREb0s5wpJ0R9Hb00Ps5Qb9+9QvvtRaQ0UyzioomfkgrzTzzTORpaYcO0VuGIzaEIzaEIx56Rwp6tJF61yZY9dXj3mvlzZUK/wG0FvuLL0Zi8WvXmlVWGgAAAHA/gaC30EPvWdvdar4v9V5rZTg92dR7jOnu3WaLFkWeUd67d+SD6mp6onDEhnDEhnDEQ28KtbW1lp2dbRkZGZaenu6VqqqqRvctLy+3adOmWbdu3bySlpZmZ8+ebZGgR+fQE6y71Vaf8sLtc/p/YTWjxkZC6nPmmK1fb3bcee83bjxQe5DPig+O2JB2hiPt3OkEPScnx/KjHmCSl5dnWRqd1ggGDBhg69at8zoBKqtWraon0C3x0N0veIJuRdvs6tjpVtF7pNlHH0VWlqEnCkdsCEdsCEc89JYhOTnZKqJGoskLT0xMbHTfrl27ekIejYcffrhFgh4Y6Zbbdmc5QX8yyTY/vcMKNtw2AAAAIAyISUFX6DwaEuyG7wVYvXq1LVu2zAoKCuz8+fO2detWm6PweCs89Btuy1vd3WqXvuaNbm/D4m70luGIDeGIDWlnBL1Lly7Nek+QiI8YMcIT8e7du3v59MtNPBKtuLjYE3OF8KPRo0cPL99x7Otjlr86wU7OWuB+82Jdg+ivPu+o+po1azr0/29OfcmSJaHmp7rsCL+21Rv+DSPfsF8vXM9cz62td3oPPSUlxU5oGpkP5d5Hjx7dKg/9ys0rtnpld9s/Ltd5/PRE4Qg/OGJDOOKhtwkS6cqoud3KoSckJDS6b2P58tbm0C/euGjvvdPD1vTOZ2o5AACAUCEmBT03N7deWFyvMzMz6+rR4XdNU1PYPfDkFy9ebFOmTGmVh155rdLWz5hk7w9aRU8UjvCDIzaEIx56W3GveejRgq739blGu6sol95UDr0pQQ+muZVfLbf3J8ywj8esDdWJwZzQ+OCIDWlnONLOnU7Q7zea8tDPXTpnH06baIUz1tMThSP84IgN4YiHHmuCHhjpbNUZ+2jyBNs/a6MBAAAAYQKC3gIP/dSFUvt4ovPQn/2Qnigc4QdHbAhHPPRYE/Qgh15a8aXtGDfeDry4NVR8yWfFB0dsSDvDkXZG0NvJQz9ZdsT2jJlgBzI/oScKR/jBERvCEQ891gQ9MNKXZZ/ZpyPHW9G8nQYAAACECQh6Czz0o+cP2IERTtAX7KEnCkf4wREbwhEPPdYEPcihH/7uUytOSbPi1/eFii/5rPjgiA1pZzjSzgh6O3noh87uts+GjLODiw/QE4Uj/OCIDeGIhx5rgh4YqeTMx/ZF8jgreeugAQAAAGECgt4CD73ozBY7OjDVPnvnED1ROMIPjtgQjnjosSboQQ698PRGOz4g1Q6tPBwqvuSz4oMjNqSd4Ug7I+jt5KF/enqDnUgaZYfXHacnCkf4wREbwhEPPdYEPTDS7tNrrbT/SDvy/gkDAAAAwgQEvQUe+s5Tq+xU3+F27MNSeqJwhB8csSEc8dBjTdCDHPonp/LsbJ8UO771VKj4ks+KD47YkHaGI+2MoLeTh771dI6d7z3Uvtpxlp4oHOEHR2wIRzz0WBP0wEhbTi2z8l5P2defnjcAAAAgTEDQW+Chbzr1plUmDrZviiroicIRfnDEhnDEQ28ramtrLTs72zIyMiw9Pd0rVVVVje7bpUuXRktLBD3IoRecXmiXegywU4cuhurEIJ8VHxyxIe0MR9q50wl6Tk6O5efn19Xz8vIsKyurWd+tqKiwpKSkVnnoG04vsCs9kuzM0cv0ROEIPzhiQzjiobcVycnJnjAHKC8vt8TExGZ9NzMz0zZv3twiQQ+MtO70PLue0Ne+O3ndAAAAgDAhJgW9W7du9eoKwTd8rzGcPXvWBg4ceM/9mvLQV5/JtOqEXlZ+ppqeKBzhB0dsCEc89LaisRz4vfLiwnPPPWc7duxo8vPi4mJPzBXCj0aPHj28fEdu6bNW072nlRR949WDBtHfjqyvWbMmVHwaqy9ZsiTU/FSXHeHXtnrDv2HkG/brheuZ67m19bjx0EtLS23IkCHN+v2mPPQVZ+dYbfcEuxiuMXH0lunRw5F2hiPtHJuCnpKSYpWVlXV15dATEhLu+p1p06ZZUVFRqwQ9MNI7p5+x29172JUrBgAAAIQKMSnoubm59cLieq3BbgEaht9PnDhhI0eObPbvN+Whv3Mqw25272XXQzYmjt4yPXo40s5wpJ075Tz0hoKemppqhw83/xnmTc1Dzy2dbtcT+tnNm+E6MZgTGh8csSHtDEfaudMJ+v1GUx56ztdT7Wr3JKupoScKR/jBERvCEQ895gQ9MNLbJybaxe6DDAAAAAgbEPQWeOjLj060C92foicKR/jBERvCEQ89FgU9yKEvPTTeyhOGho4v+az44IgNaWc40s4Iejt56MtKxtu5hOH0ROEIPzhiQzjioceioAdGWrx/gp1JGGUAAABA2ICgt8BDX7xnvJ3qkUpPFI7wgyM2hCMeeiwKepBDX7xjvJX2HBc6vuSz4oMjNqSd4Ug7I+jt5KG/uXW8new1gZ4oHOEHR2wIRzz0WBT0wEhvbhlvX/WaZAAAAEDYgKC3xEMvGG/H+0ylJwpH+MERG8IRDz0WBT3Iob/53gQ70nd66PiSz4oPjtiQdoYj7Yygt5eHvma8He43g54oHOEHR2wIRzz0WBT0unnoKyfYof4ZBgAAAIQNCHoLPPQleRPss6TZ9EThCD84YkM44qHHoqAHOfQlyydYyYDnQseXfFZ8cMSGtDMcaWcEvZ089KVLJ9rBQc/TE4Uj/OCIDeGIhx6Lgh4Y6a1FE604+SUDAAAAwoaYFPTa2lrLzs62jIwMS09P90pVVVWT+588edImTJhg3bp180prPfTlCyda0ZAseqJwhB8csSEc8dDbAzk5OZafn19Xz8vLs6ysxoX23LlzlpSUZCUlJa320IMc+tvzJ9qBYa+E7sQgnxUfHLEh7QxH2rnTCXpycrJVVFTU1cvLyy0xMbHRfSX0u3btatHvN+Wh5748yQ6MeJWeKBzhB0dsCEc89PZAw7C5QvBNhdIl9BL1Rx55xNtnypQpdw3PNybogZHyXppkhaMWGgAAABA2xKSgd+nSpVnvCQ899JAVFhZ6ol9TU2PLli2ziRMnNrpvcXGxJ+YK4Udj4MCBXngk/3kn6GPe9HpWqgc9rI6u79mzJ1R8GquvWbMm1PxUlx3h17Z6UMJ8Pob9euF65npubb3Te+gS9Ib7du3atUUeepBDf/e5yVaYtjR0vTLyWfHBERvSznCknTudh56SkmKVlZV1deXQExISGt1XYhydbxcee+yxFgl6kENfNcsJ+oTloTsxyGfFB0dsSDvDkXbudIKem5tbLyyu15mZmXX16PC7Pps3b15d3ryoqMgWLFjQIkEPjLR2xmQ7MOUdAwAAAMKGTjkPvWE+/bXXXvMGxSnULnG/detWqzz09dOdhz59JT1ROMIPjtgQjnjosYCmcugbpk62opmrQ8eXfFZ8cMSGtDMcaWcEvY2CLg+91m0bJ06yotnr6YnCEX5wxIZwxEOPRUGXkWrctiXNCfrzBQYAAACEDQh6Mz306tpq2zpmoh3M/JCeKBzhB0dsCEc89FgUdOXQr9+6bttHT7Til7eGji/5rPjgiA1pZzjSzgh6O3joV29ctd0jJ1jJgk/oicIRfnDEhnDEQ49FQZeRLn9/2T4d5gR94W4DAAAAwgYEvZkeetXVKiscOt4+e/NTeqJwhB8csSEc8dBjUdC95WMvl9vBp8bZ58sOhI4v+az44IgNaWc40s4Iejt46GUXv7PPk8fZobcP0hOFI/zgiA3hiIcei4IuI31X9Y19MXCMHV55yAAAAICwAUFvpod+uuIrOzpgjB1ZfZieKBzhB0dsCMf49tBXr15tsYDGcujfVhyxr/qPsmMbjoeOL/ms+OCIDWlnONLOoRF0PQXt97//fb0no8WKh/51+ef2db+RdnzjCXqicIQfHLEhHOPbQ3/sscc8UVeZP39+zAi6jPTV+SL7ps9w++qjUgMAAADChgeeQ9+yZYt169bNE3U9o/zzzz+PCQ/9+Ll9drr3MDu5/RQ9UTjCD47YEI7x7aFHQ/n0hx56qM5jD4reC5ugK4d+5Ltd9l2voVa652zoTgzyWfHBERvSznCknUMp6Hl5efVEPCgPP/xwKD30Q2c/sbLEp+zbwvP0ROEIPzhiQzjioW/evLku5P7oo4/asWPHQmfoxnLon539yC4kDrbTJRUGAAAAhA0PVNAff/zxOq/8jTfeaPXv1NbWWnZ2tmVkZFh6erpXmho53zCkH5SWeujFZzbaxR4D7OyRi/RE4Qg/OGJDOMa3hy4hTUxMtEuXLrWJdE5OjuXn59fVFb7Pyspq8v9sq4euHHrhmfV2pUeSlZ24HLoTg3xWfHDEhrQzHGnn0Aj6xo0b24V0cnKyVVTcCX2Xl5d7HYX7Jejy0PedWWfXE/paxanr9EThCD84YkM4xreH3l5QDj4aCsE3fC9a0DXQTqV79+62cOFCq66ubpGgy0h7Tr9r1Qm9rOp8tQEAAABhQ0wKemNed3M8cXn1EvSZM2c2+nlxcbEn5grhR2PgwIG2pnie1XTvaZcqa7yelcIlQQ+ro+t79uwJFZ/G6mvWrAk1P9VlR/i1rR6UMJ+PYb9euJ65nltb7/QeekNo365du7bIQ1cOffvpt622e4JdvRq+Xhn5rPjgiA1pZzjSzp3OQ09JSbHKysq6unLoCQkJzfrutWvX7He/+12LBF059I+/Wex56DduhO/EIJ8VHxyxIe0MR9q50wl6bm5uvbC4XmdmZtbVo8PvqampVlhY6HnmEnNNd7vXlLnGcuhbS9+wm9172c2bBgAAAIQOMSno95qHHi3o69evt9GjR3ur0D3xxBO2ePHie/5+Yx761hML7Hr3fu7/picKR/jBERvCEQ89JtBYDn3L8Wy70v3JUPIlnxUfHLEh7QxH2hlBb6Ogy0Pf+EWWXUoYRE8UjvCDIzaEIx56rAq6jPTBwZfsQkKyAQAAAGEEgt5MD31D4YtWljCUnigc4QdHbAhHPPRYFXTl0N/b+7yd6zEslHzJZ8UHR2xIO8ORdkbQ28FDX7dzjp3pMZKeKBzhB0dsCEc89FgVdBlpzdZn7dvEVAMAAADCCAS9mR76qi2zrTRxDD1ROMIPjtgQjnjosSroyqGvKnjGTvYeH0q+5LPigyM2pJ3hSDsj6O3gob+7fpZ92XsiPVE4wg+O2BCOeOixKugyUv6amXas72QDAAAAwggEvZkeev7KDDvSdyo9UTjCD47YEI546LEq6Mqhr8ybaV/0nx5KvuSz4oMjNqSd4Ug7I+jt4aHnzLDPk9LpicIRfnDEhnDEQ49VQfdy6MtmWsmADAMAAADCCAS9mR76yjdnWPGg2fRE4Qg/OGJDOOKhx6qgezn019OtaPCzoeRLPis+OGJD2hmOtDOC3g4e+rsLnrYDQ56nJwpH+MERG8IRDz1WBV1Gejc73QqHvmAAAABAGBGTgl5bW2vZ2dmWkZFh6enpXqmqqrrn9zZt2mRdunRplYe+KvNp2z/sJXqicIQfHLEhHPHQ2ws5OTmWn59fV8/Ly7OsrKy7fufgwYM2efLkVgm6cuhrXnza9o14OZQnBvms+OCIDWlnONLOnU7Qk5OTraKioq5eXl5uiYmJTe5/6tQpGz16tF2/fr3VHvqaOdPt09Hz6InCEX5wxIZwxENvL3Tr1q1eXSH4hu8FUCg+JSXFKisrvXprBF1GWvesE/Qx8w0AAAAII2JS0BsT5aaEeuzYsXby5Ml77icUFxd7Yq4QfjQGDhxo6zOm225f0NWzUrgk6GF1dH3Pnj2h4tNYfc2aNaHmp7rsCL+21YMS5vMx7NcL1zPXc2vrnd5Dl4A3VlrioSuHviF9uu2d8Hooe2Xks+KDIzakneFIO3c6Dz06hC4oh56QkNBq7/5egq4cesF0J+iT3wzliUE+Kz44YkPaGY60c6cT9Nzc3Hphcb3OzMxslmi3Noe+cfI02zt9mQEAAABhRKech97egi4P/cNJ0+zT9OX0ROEIPzhiQzjioccKGsuhb0mbavtmvRNKvuSz4oMjNqSd4Ug7I+htFHR56NvGTrF9z66kJwpH+MERG8IRDz1WBV1G+mT0ZNv/4moDAAAAwggEvZke+s6Rk+1A5nv0ROEIPzhiQzjioceqoCuHvnv4JCuaVxBKvuSz4oMjNqSd4Ug7I+jt4KHvTZloBxd+SE8UjvCDIzaEIx56rAq6jLR/yAQrWbTVAAAAgDACQW+mh140OM0OvbWTnigc4QdHbAhHPPRYFXTl0EsGpdnhd/aGki/5rPjgiA1pZzjSzgh6O3johwaMtWOriuiJwhF+cMSGcMRDj1VBn/70dDuSNMaOv/eZAQAAAGEEgt4MQR8xaoQd7z/avvrgCD1ROMIPjtgQjnjosSro/fr3sxN9R9nJLV+Gki/5rPjgiA1pZzjSzgh6GwV9+Ijh9k2fEVb6SSk9UTjCD47YEI546LEq6JOmTLRTvYfZt7tPGQAAABBGIOjNEPRhw4fYd72G2un9Z+mJwhF+cMSGcMRDj1VB75fUy8p6JtvZkrJQ8iWfFR8csSHtDEfaGUFvo6APHT7ILvQcbOcPV9AThSP84IgN4YiHHquCPm7CSLvYY4CVf3XRAAAAgDAiJgW9trbWsrOzLSMjw9LT071SVVXV6L6rVq2ykSNHWteuXa1bt242ceJEO3XqVIsEfcjwJ+1KQpJd+OYKPVE4wg+O2BCOeOjthZycHMvPz6+r5+XlWVZWVqP7pqamWmFhodcJUFm9erX17du3RYLed0B3u57Q1y5+dz2UJwb5rPjgiA1pZzjSzp1O0JOTk62i4k4+u7y83BITE5v9/YcffrhFgp48rK9Vd+9tlyuq6YnCEX5wxIZwxENvLyh0Hg153g3faww1NTW2du1aLwTfGIqLiz0xl8cfjcHDeltN95527PDJuoZQ7ypoEOrUqVOnTr2j6zEp6F26dGnWew0/V1G+/dKlSy3y0AcP72W13RPsxg16onCEHxyxIRzx0DvcQ79165Y3SG7EiBEtEvQ+A3/neejOwQ8lyGfFB0dsSDvDkXbudIKekpJilZWVdXXl0BMSEpr9fY14b4mgDxra3cuh0xOFI/zgiA3hiIfejsjNza2X59brzMzMunp0+D0tLc1KSko8L/7GjRteDn3KlCktEvShI3rZ9e79DAAAAAgrOuU89GhB37Rpkzd1TSPbH3/8cW9627Vr11ok6AMGP2FXuj9JTxSO8IMjNoQjHnosoaGg9+73mF1KGBhavuSz4oMjNqSd4Ug7I+htFPSkAb+zCwnJ9EThCD84YkM44qHHsqAPHPSElSUMMQAAACCsQNCbIeh9+z5u53oOoycKR/jBERvCEQ89lgU9sedv7UzPEaHlSz4rPjhiQ9oZjrQzgt5GQe+T+Kh9mzianigc4QdHbAhHPPRYFvR+vR+30l6pBgAAAIQVCHozBL13wm/tRO9x9EThCD84YkM44qHHsqD3fKKbfdl3fGj5ks+KD47YkHaGI+2MoLdR0Hv9/hE72m8iPVE4wg+O2BCOeOixLOi9n/itHe4/2QAAAICwAkFvjof+P93s86Sp9EThCD84YkM44qHHsqD3fLyrfTZgemj5ks+KD47YkHaGI+2MoLdR0BP/v652cODToeVLb5kePRxpZzjSzgh6MwS912P/z4oGzzQAAAAgrEDQm+Oh//ZhK3wqg54oHOEHR2wIRzz0mBb0br+x/UNnh5Yv+az44IgNaWc40s4IehsFvdf/+419OuxZeqJwhB8csSEc8dDbE7W1tZadnW0ZGRmWnp7ulaqqqkb3Xb16tY0ePdq6du1qjzzyiPed8vLylgl619/Y3hFzDAAAAAgrYlLQc3JyLD8/v66el5dnWVlZje6bmppqhYWFXiegpqbGli9fbkOGDGmRoPf+zW9sz6gX6InCEX5wxIZwxENvTyQnJ1tFRUVdXR53YmJis7//8MMPt0zQ//sh2506N7QnBvms+OCIDWlnONLOnU7Qu3XrVq8u77vhe02hqKjI89pbJOhdHrJd416iJwpH+MERG8IRD7090aVLl2a91xA7duywwYMHN5lDLy4u9sRcIfxo9Pmv/7ZPxsytawD9Ve+KOnXq1KlTD0s9bjz0xYsX25QpU5ocPHc3D73vr//bdk6aR08UjvCDIzaEIx56eyIlJcUqKyvr6vK4ExISmtz/4MGD3sC45uIHgv5/naBPeSW0Jwb5rPjgiA1pZzjSzp1O0HNzc+uFxfU6MzOzrh4dft+7d2+zvPK7CXq//+xiO59+lZ4oHOEHR2wIRzz09sS95qFHC7peN1ZaIuj9/6OL7Zi50AAAAICwgpXimiHoSb/8L9v5zCJ6onCEHxyxIRzx0GNZ0J/8hRP0ZxeHli/5rPjgiA1pZzjSzgh6GwV9wL87QX9hKT1ROMIPjtgQjnjosSzoA3/+a9v90nIDAAAAwgoEvRmCPujfnKC//A49UTjCD47YEI546LEs6Mk/+5XtfTU/tHzJZ8UHR2xIO8ORdkbQ2yjoTzlB/3ThKnqicIQfHLEhHPHQY1nQh/zLf9q+xesMAAAACCsQ9GYI+lAn6AeWv09PFI7wgyM2hCMeeiwLeso//4cVv7MptHzJZ8UHR2xIO8ORdkbQ2yjow3/6S/vs3Y/oicIRfnDEhnDEQ49lQR/x01/Yofe2GwAAABBWIOjNEPRR//QLO/LBHnqicIQfHLEhHPHQY1nQR//k3+34lsLQ8iWfFR8csSHtDEfaGUFvo6CP+cm/2cntn9EThSP84IgN4YiHHsuCPvYf/82++fSIAQAAAGEFgt4MQU/7x5/Z2YMn6YnCEX5wxIZwxEOPZUEf7wS97Mjp0PIlnxUfHLEh7QxH2hlBb6OgT/zxv1jl1+X0ROEIPzhiQzjiobcnamtrLTs72zIyMiw9Pd0rVVVVd/1OaWmpPfroo60S9Ek//me7dOayAQAAAGFFTAp6Tk6O5effeZxpXl6eZWVlNbl/ly5d6kprBH3KP/yTXau4Tk8UjvCDIzaEIx56eyI5OdkqKirq6uXl5ZaYmHjP77VW0Kf+w0/s+0vVoT0xyGfFB0dsSDvDkXbudILerVu3enWF4Bu+156CPv0f/tFufl9DTxSO8IMjNoQjHnp7ojFhbo5Y32uf4uJiT8wVwq9npH/4sW3d+nFdA+ivelfUqVOnTp16WOp46M300OmJwhF+cIQfHPHQ2xkpKSlWWVlZV1cOPSEh4b4JunLoYQb5rPjgiA1pZzjSzp1O0HNzc+uFxfU6MzPznsLd6lHuP/4pPVE4wg+O8IMjHnp7417z0BsKd/S0teZMX2tsHjoAAAAQZrBSXDMEXSvF0ROFI/zgCD844qHHuKCP//H/CTVf8lnxwREb0s5wpJ0R9DYKetqP/5WeKBzhB0f4wREPPdYFXc9DBwAAAMIMBL0Zgj7mH39OTxSO8IMj/OCIhx7rgj76J/8ear7ks+KDIzakneFIOyPobRT0UT/5BT1ROMIPjvCDIx56rAv6yH/6pQEAAABhBoLeDEEf8dP/oCcKR/jBEX5wxEOPdUEf9tP/DDVf8lnxwREb0s5wpJ0R9DYKeso//4qeKBzhB0f4wREPPdYFfei//MoAAACAMANBb4agD/k/v6YnCkf4wRF+cMRDj3VBfyrkgk4+Kz44YkPaGY60M4LeRkFP/tf/oicKR/jBEX5wxEOPdUEf9K9dDAAAAAgzEPTmCPq/hVvQ6S3To4cj7QxH2hlBb4agD/j5f4eaL/ms+OCIDWlnONLOCHpbBf3fwy3o9Jbp0cORdoYj7RyTgl5bW2vZ2dmWkZFh6enpXqmqqmrzvk0J+pO/fMgAAACAMCMmBT0nJ8fy8/Pr6nl5eZaVldXmfZsS9KRf/oaeKBzhB0f4wREPvb2RnJxsFRUVdfXy8nJLTExs875NCXr//wi3oJPPig+O2JB2hiPt3OkEvVu3bvXqCqs3fK81+zYp6P/5MD1ROMIPjvCDIx56e6NLly7Neq+l+xYXF3tivn79+nrvjxs3zjMUhUKhUChhLaNHj8ZDBwAAAOINoRD0lJQUq6ysrKsrL56QkNDmfQEAAAAE/QEiNzfXG60eQK8zMzPr6tEh9XvtCwAAACDoHYR7zS2PFvTWzENviHXr1nm59bAWdVLCzA+O2BCO2BCO95/fwYMHY0/QHzQajnqHHxzhB0dsCMdY5xeXgt7SXg/84IgN4YgN4Rh2fnEp6AAAAEBnA4IOAAAAIOgAAAAAQNABAAAAgKC3BO0x3e1+obS01B599NHQ8V29erW39GDXrl3tkUce8bhoIZ8w2XPVqlU2cuRIj6NWDJw4caKdOnUqtG2+adOmdp+G2VaIT2MlbDY8efKkTZgwwWvn6NUhw8Ix7HbUtTtt2rQ6+6WlpdnZs2dDZcOamhqbN2+ed7/RPXHBggWhaeeW3qc7gm/cCHprHrv6oG8CYeObmppqhYWF3ompC2358uU2ZMiQUNkzmqOKOiF9+/YNZZtrFOzkyZPrtXUYODb1LIQw2fDcuXOWlJRkJSUlMXNt66mQ4hwWjgMGDPDW4AiuFXWG+/fvHyobvvbaa1ZUVOS9vnTpks2fP9+WLVvW4Rxbc5/uCL5xI+iteexqR95Uw8r34YcfDr09w8hRUQNFO65fv16vrcPA8W6CHhYb6ka4a9eumLq2tYLl5s2bQ8NRUSwJeZivlYYesDzaoIMexmvlbpw6gm/cCHrYH+rS8EQJI1/1nOURh5Wfoghr1671QvBh4qibUvQzCKLbOgwcxUc3dpXu3bvbwoULrbq6OlQ21I1Qoq5QrP7/KVOm1IUvw3guKpQ9cODAULWzolfydgsKCuz8+fO2detWmzNnTqhs+NBDD/2g06GOSJiulebepzuCb9wIekseuxoGfmHju2PHDhs8eHBdDj1s/IJwmPJUCtWFiePYsWO9/G9jHMJmR3kUEvSZM2eGip9u9NHpHwmTxkuE9dp+7rnnvGsmTO0sER8xYoQn4uq4KZ9++fLlUNlQbaowtdpYbb1///46QQ8Dx5bcpzuCLx46Hvo9sXjx4noeUVjteevWLS8vqJtWmDjebbBUGO0oDmHyigJBDzvHABo8FYw1CdP1oijRiRMn6uoSzuB522GKZqmjEQzEVdqiT58+eOgI+g9P5jA/drXhiRIWvhrIJc8oluwZ3OjDyjG6rcPI8dq1a/a73/0uVPw0eCs6Hyk89thjobShBCkY2BWmdg7y5Y29F9ZrZe/evd5I8bBwbMl9uiP4xo2gh/2xqw1PlDDw1cXU1DSLsNhTU2808lm93xs3bng5dEUTwtzmYXsccPRMAYm5bqBvvPFGqGyo/1fTmYLzUYIZTGkKUzvLAw7GcITtetG1orB74C0GkbewXiuHDh3yRuYHab4wcGzJfboj+DIPPQQ397DOW42F+cma1y1Bkqfx+OOPewOnJEphb/MwnZfr16/3Qq8Kaz/xxBPejT6M142mNCkMqwiMxF0plrBx1Ll4+PDhUN5/9P/p/5X9VJRLD3LoYbJhwE+2jB570pEcW3MfZB46AAAAABB0AAAAAEEHAAAAAIIOAAAAAAQdAAAAAAg6AAAAgKADAAAAAEEHAAAAAIIOAAAAAAQdAPBgsWXLFm+td63W1hi0Wp8+134AAAQdABBiaBlWLX+5cuXKeu9r/Wq9r0ewAgAQdABADEBrU0u8d+/e7dW3bt1a90x6AACCDgCIEejhKHr2vJ7xrKfc6e+wYcOsuroa4wCAoAMAYgmXLl2yxMREzzPv2bNnKJ5mBwCCDgAALYQEHEEHAEEHAMQwgpC7nl2twXGE3AFA0AEAMYjp06d7nvm2bdu8+o4dO7y63gcAIOgAgBhAMG1N09SiIU9d7+tzAACCDgAIMYKFZbSATGPQgjMsLAMAgg4AAAAABB0AAABA0AEAAACAoAMAAAAAQQcAAAAAgg4AAAAg6AAAAABA0AEAAACAoAMAAAAAQQcAAAAAgg4AAAAg6AAAAABA0EHrGu1HP6JQKJT7WgCCDh6QoAMAAPcYgKBzsQEAAPcYBB1wsQEAuMcABB1wsQEAuMcABB1wsQEAuMcABJ2LrZXo0qVLp7Ypx8fxcXwIOoIOEPQW7FdbW2vZ2dmWkZFh6enpXqmqqoqZG2Zz9ystLbVHH3005gThXvutXr3aRo8ebV27drVHHnnEa8fy8vJOc3yrVq2ykSNHesfXrVs3mzhxop06darTnZ/Cpk2bmtwfQUfQAYJ+z/1ycnIsPz+/rp6Xl2dZWVmd6oapfYLS2QQ9NTXVCgsLvY5ZTU2NLV++3IYMGdIpj09FHZi+fft2OkE/ePCgTZ48GUFH0EFnFXRd3LqB9ezZ0/NOpkyZYpcvX653o9BNfN68eZ53piJvW+81FLKmbhTJyclWUVFRV5d3l5iY+MBuivf7+FrrLcXi8QV4+OGHOb4YOj5FHBRluX79OoKOoINQCfq2bWbp6a0r+m6DG0pubq5du3atzjvJzMysd0NZuHChrVy5ss6DWbt2rb322mvNFjHdqKKh32j4XjTa8fAeyPG1VNC3uS29lZu+25HHJxQVFXle7d2O0LVGK0vHHp+EUt9XCP4BHN4DOT6lt1JSUqyysvKu+yPoCDroCEEvKzM7dKh1Rd+9iwDphhbkgYPPfv/739d5BMKtW7fsiSeeaPYNpbHP7/addjy8B3J8LRX0MrcdauWm73bk8e3YscMGDx58jxy6OB5qZem44wu8XI3xuHTp0oM4vAdyfGPHjrWTJ0/ec38EHUEHHSHo7RzyayrcGHx2t33uh4cea8fXUkGP1eNbvHixFxJ+kAMaH3T7SSw1SG7EiBGd5vgahuWbCs8j6Ag66GSCLk/gf/7nf+p9Jm+goYcQ7NOcG0p0uE+Qd5eQkNAhN8z7cXxhEvT7dXwaUKWBYw8aD7r9AmjEe2c9Pjx0BB10YkEPwnHynN9//32bP39+vQtfOTyNTA9yeMrnRefwHn/88btO81GOUN8PoNdBnvBB3DDv9/F1tKDf7+Pbu3dvh00zfBDHl5aWZiUlJd53b9y44eWoFYnojOcngo6gg04u6BrgFMwx1mja6urqehe+vAONrNU+KnotLyFAQUFB3QjcxtCR89AfxPEFv9XaEdVhP77mhmxj9fg0N1v/h8LYEkdNqdQgtc50fiLoCDqIE0HvzOD4OD6Or2PvMQBBBw/oYntQucKOAsfH8XF8CDqCDuJC0AEAgHsMgg642AAA3GMAgg642AAA3GMAgh6HFxuFQqHczwIQdAAAAAB0AP5/VcW9jFJjHboAAAAASUVORK5CYII=",
      "text/plain": [
       "BufferedImage@2c3ca7aa: type = 2 DirectColorModel: rmask=ff0000 gmask=ff00 bmask=ff amask=ff000000 IntegerInterleavedRaster: width = 500 height = 400 #Bands = 4 xOff = 0 yOff = 0 dataOffset[0] 0"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "plot(rocs(?,0->5))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": []
    },
    {
     "data": {
      "text/plain": [
       "0.92796,0.95949,0.97780,0.96988,0.98077,0.94405,0.97540,0.97547,0.97647,0.94959,0.97718,0.94474,0.86756,0.91574,0.98457,0.96938,0.97628,0.95293,0.91328,0.95620,0.90527,0.91562,0.95315,0.94977,0.97943,0.95953,0.96210,0.95130,0.97605,0.94854,0.94699,0.96866,0.94554,0.96259,0.91478,0.95784,0.90442,0.96816,0.80705,0.94270,0.94160,0.95267,0.66955,0.92352,0.95091,0.92467,0.90882,0.95015,0.96943,0.96349,0.97251,0.97540,0.91864,0.95583,0.93168,0.96431,0.96347,0.61410,0.91780,0.96541,0.98574,0.91007,0.96804,0.76783,0.94777,0.87311,0.95931,0.98108,0.90297,0.98872,0.92896,0.96976,0.88119,0.88600,0.90381,0.93929,0.92118,0.90963,0.96243,0.95381,0.96487,0.88981,0.91170,0.92376,0.93833,0.87769,0.74387,0.93540,0.95893,0.98602,0.95531,0.93069,0.96342,0.97120,0.95280,0.78630,0.69109,0.66667,0.44554,0.98515,0.50000,NaN,NaN"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "val aucs = mean(rocs)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": []
    },
    {
     "data": {
      "text/plain": [
       "0.9753971348784144"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "aucs(6)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We could go ahead and start tuning our model, or we automate the process as in the next worksheet."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "name": "scala",
   "version": "2.11.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}
